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Preface! 


The naturalness or artificiality of faces exists on a continuum rather than 
as distinct categories. While biological faces are influenced by cultural 
and societal factors, face representations always maintain a connection 
to their underlying biological foundation. However, throughout history, 
humans have been fascinated by the concept of creating completely artifi- 
cial faces, devoid of any human intervention. This enduring idea dates back 
to ancient times. Various manifestations of this concept include spontane- 
ous depictions of faces in nature, now classified as instances of pareidolia, 
facial images perceived as originating from transcendence, and the artistic 
renditions of faces. Today, we are captivated by the utopian vision of pho- 
torealistic digital portraits, generated through the cutting-edge technolo- 
gies of deep learning and artificial intelligence (AI). This interdisciplinary 
anthology comprises a collection of chapters that explore diverse tradi- 
tions, facets, and indicators of humanity’s utopian pursuit of the creation 
of an artificial face. 

The proposed collection emerges from the interdisciplinary intersection 
of semiotics, neurophysiology, cognitive psychology, visual history, digital 
art, and AI. It stems from the extensive research conducted in the frame- 
work of the ERC Consolidator Grant project titled FACETS: “Face Aes- 
thetics in Contemporary E-Technological Societies”. Delving into the realm 
of artificial faces, the collection addresses a specific inquiry. The concept 
of nonhumanly crafted facial images dates back centuries. Firstly, human 
beings display a neurophysiological inclination toward pareidolia, which 
refers to the tendency to perceive faces in natural surroundings such as 
tree trunks and clouds. Additionally, ancient sources across various cul- 
tures chronicle prodigious appearances of facial images in stones, gems, 
landscapes, and more. Religious contexts often attribute these miraculous 
facial images, known as “acheiropoietai”, to manifestations of deities. The 
collection comprehensively explores this captivating tradition while draw- 
ing comparisons to contemporary trends in the development of “transhu- 
man portraits”. This exploration encompasses the technological realm, 
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including generative adversarial networks and robotics, the medical field 
with a focus on aesthetic surgery and face transplantation, as well as the 
arts, particularly highlighting the thought-provoking creations of modern 
digital artists who delve into the realm of “artificial faces”. 

The face, being an immensely intricate and multifaceted entity, has con- 
sistently captivated the attention of societies and cultures, as evident in 
their texts, representations, customs, and laws. Its profound significance as 
a social and cultural cornerstone renders it impossible to fully encompass 
its complexity through a single overview. Consequently, the chapters con- 
tained within this collective volume have been carefully selected to focus on 
a more delimited theme. This theme explores the notion that the face, the 
visage, can be an outcome not only of natural creation but also of artificial 
creation ex nihilo. The roots of this idea can be traced back to ancient 
times, such as the biblical depiction of the human face, formed from shape- 
less clay, created in the image and likeness of the divine. Over the course 
of Western cultural history, this notion has branched out into intricate 
filiations, culminating in the contemporary aspiration to generate digital 
facial images autonomously through machines with no inherent need for 
biological or ontological correspondence to the face. Each chapter in this 
collective volume engages in thoughtful contemplation of this issue, which 
holds strong ethical implications. The detachment of a disruptive image 
like the face from its corporeal substratum, transforming it into a pure 
simulacrum constructed by machines, raises pertinent questions and calls 
for careful examination. 

In the first chapter, Marco Viola studies the biological and social func- 
tions played by the face (or by the organ it hosts), tracing them back to their 
evolutionary history. The face turns out to be functionally overcrowded. 
Then, drawing on some notions from the philosophy of artifacts, the chap- 
ter sketches a theory and a taxonomy of masks. Masks are construed as 
artifacts whose function is to interact with some of the functions of the 
face. They are sorted into different categories, depending on which func- 
tions of the face they are intended to act upon. Viola also considers some 
interesting phenomena that complicate functional attributions. These con- 
siderations are then applied to a specific type of mask, namely the medical 
face mask. 

In the second chapter, Gabriele Marino delves into the emotional state of 
frustration brought about by the outbreak of the new coronavirus/COVID- 
19 pandemic in December 2019. In particular, it examines the iconic 
Internet meme of Captain Picard from Star Trek performing a facepalm, 
wherein he covers his face with his left hand as a gesture of exasperation. 
This meme serves as a cultural sign that epitomizes the prevalent senti- 
ment. The chapter adopts a semiotic perspective to explore the pandemic 
as a form of discourse, encompassing the concept of the “infodemic”, the 
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dynamics of online virality, and the role of Internet memes. It also examines 
the transformative effect of the pandemic on two body parts traditionally 
considered ‘natural’: the face and hands. The notion of “parafaciality” is 
introduced, referring to the transformation of the face into a cultural inter- 
face through various accessories. Through the collaborative involvement 
of hands, the face’s inherent naturalness is called into question, reshaping 
it into a semiotic mask capable of visually and haptically (through touch) 
embodying the spirit of the times (Zeitgeist). However, in the “time of the 
virus”, one was compelled to refrain from touching one’s face to prevent 
contagion, which means that one had to relinquish the iconic facepalm ges- 
ture. This deprivation would be doubly frustrating, as people were unable 
to express their frustrations as fully as they would wish. 

In the third chapter, Cristina Voto delves into the intricate sociocultural 
dynamics of artificial intelligence (AI) systems and their role in conveying 
the concept of otherness. The focus is on the impact of automated facial 
images in shaping the sociocultural communication of otherness, as the 
face and its representation through visual artifacts serve as powerful tools 
for displaying and defining identities. To accomplish this goal, the chap- 
ter proposes a comprehensive exploration of Emmanuel Lévinas’ seminal 
work on the phenomenology of the face (1961), complemented by Judith 
Butler’s insightful interpretation of his ideas (2004). By bridging these phil- 
osophical perspectives with contemporary visual culture, Voto establishes 
an epistemic framework that enables one to navigate the realm of facial 
images. Within this framework, the chapter uncovers a range of processes 
that encode the notion of otherness, encompassing everything from media 
portrayals of adversaries to the proliferation of para-facial expressions 
in the modern era. Moreover, recognizing the arts as a privileged domain 
for reverse engineering and reconfiguring established patterns, the chapter 
focuses on the analysis of an artistic practice that sheds light on how other- 
ness is effectively communicated within AI systems. 

In the fourth chapter, Enzo D’Armenio introduces the concept of a 
“dramaturgy of the face” to elucidate the nature of images and videos 
shared on social networks. The analysis begins by examining the mecha- 
nisms underlying the construction of value, which involves the accumula- 
tion and transformation of quantitative appreciation expressed through 
actions such as likes, follows, and shares into monetary and reputational 
value. Subsequently, the focus shifts to the experiential and algorithmic 
pressures that shape identity, necessitating the management of these pres- 
sures through the use of image-based identity narratives. Lastly, the study 
explores the relationship between the semiotic qualities of the face and 
those of images by comparing two classical genres with two genres preva- 
lent on social networks. On the one hand, it compares the artistic self- 
portrait canon to selfies and identity images shared on social platforms. 
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On the other hand, it contrasts the transformation of actors into fictional 
characters with the performances of streamers on Twitch. 

In the fifth chapter, Maria Giulia Dondero explores five distinct theoretical 
attitudes toward portraiture, each representing a different historical period 
and artistic perspective. These attitudes include (i) the Renaissance paint- 
ing tradition, (ii) the Baroque era tradition, (iii) Francis Bacon’s approach 
to depicting heads in portraits, (iv) Roland Barthes’ conception of pho- 
tographic portraits as discussed in his seminars at the Collége de France, 
and (v) the utilization of contemporary facial images for creating deepfake 
videos. These theoretical attitudes serve as performative manifestations 
of different portraiture ideologies and are exemplified through renowned 
portraits. The main objective is to investigate the evolution of the figure- 
ground relationship over centuries and analyze how these five types of 
portraits mold and remodel the interplay between the entirety of visual 
composition and its individual elements. Through this analysis, insights 
are gained into the connection between portraits and diverse notions of 
individuality. 

In the sixth chapter, Antonio Dante Santangelo focuses on analyzing the 
portrayal of the protagonists’ faces in three films: La vie d’Adéle — Chapi- 
tres 1 & 2 (Kechiche, 2013), Jeune femme (Serraille, 2017), and Chiara Fer- 
ragni — Unposted (Amoruso, 2019). These films serve as symbols reflecting 
the contemporary mindset, particularly within the Italian cultural context. 
Each of these works holds significance as they embody two contrasting 
cultural models: the vagabond and the self-made individual. The utilization 
of these cultural models and their implications shed light on the evolving 
strategies that are employed to attribute meaning to existence. Rather than 
perceiving meaning as a predetermined path or a life project to relentlessly 
pursue, it is now seen as something discovered retrospectively, interwoven 
within the multitude of experiences one encounters. To support this argu- 
ment, the analysis extends beyond the films themselves to encompass their 
reviews and connections with other texts, providing a broader contextual- 
ization within the current cultural landscape. 

In the seventh chapter, Bruno Surace delves into the concept of a unique 
type of degenerate face, characterized by its empty and vacant nature, 
devoid of any discernible features. The analysis begins by examining 
the phenomenon of Slender Man, an Internet-born urban legend that 
originated from a “creepypasta” and has evolved into a powerful digital 
imagery icon, spawning countless textual variations and even inspiring film 
adaptations. By studying Slender Man, whose face/non-face represents its 
most distinct attribute, one can analyse the significance of the face within 
the realm of viral and memetic communication. Despite being effectively 
nullified, Slender Man’s face encompasses a multitude of disparate mean- 
ings. Drawing from this case study, the chapter outlines a specific theory 
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concerning the extreme limits of the face as an empty surface, drawing 
comparisons between Slender Man and similar figures in literature, such as 
Marcel Schwob’s Sans-gueules, as well as exploring the presence of feature- 
less faces in audiovisual and cinematic works. This analysis culminates in 
the development of an initial repertoire of films featuring characters with 
no discernible facial features, emphasizing the existence of significant cul- 
tures of the non-face alongside cultures of the face. 

In the eighth chapter, Remo Gramigna wonders how aliens look like 
and whether they have a face. The chapter explores “imaginary faces” by 
unpacking a plethora of sources, from media studies to conspiracy theo- 
ries, including cultural anthropology, as well as semiotic theory (sign-based 
theories of semiosis). The topos of the alien, thought of as a form of extra- 
terrestrial life, has a long pedigree and an enigmatic status. This is a very 
complex and controversial subject, almost without limit. Today, Gramigna 
claims, the archetype of the alien has seen a reemergence through the wide- 
spread use of digital media and the massive proliferation of “conspiracy 
theories”. Such matters are at the forefront of discussion, arousing inter- 
est among experts and ordinary people alike. In some of these theories, 
the existence of extraterrestrial entities is supported, and the present, past, 
and future role of alleged “alien races” in the evolution of human beings 
is discussed. These accounts pay attention to the outward characteristics 
of extraterrestrials and their physical appearance, including facial features, 
to identify them and distinguish one from the other. One could, therefore, 
speak of a phenomenology of the alien which is traceable in contemporary 
conspiracy discourse at a crossroads between ufology, alternative history, 
anthropology of aliens, and personal accounts of “abduction experiences”. 

In the ninth chapter, Gianmarco Thierry Giuliana focuses on the semi- 
otic analysis of the video game Nier: Automata, specifically examining how 
the various occurrences and forms of digital faces attached to protago- 
nist figures contribute to the attribution of personhood and even human- 
ity to androids, robots, and drones. The central theme explored in the 
chapter is the contemporary trend of extending the semantic category of 
“human” to nonhuman agents with a particular emphasis on the rhetorical 
function of faces and digital technology in this process of transhumanist 

re-semantization and revalorization. By delving into Yoko Taro’s tran- 
shumanist narratives and the rhetoric employed in the portrayal of digital 
faces, Giuliana uncovers how the contemporary inclusiveness of the human 
category reflects a transcendental understanding of the face, which in turn 
connects to a transcendental and supra-individual concept of subjectivity. 
Ultimately, the conclusion emphasizes the pivotal role of digitizing faces 
in acknowledging this transcendental dimension, highlighting the unique 
nature of digital faces in video games as both masks and prosthetic devices. 
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In the tenth chapter, Silvia Barbotto reflects on the contemporary expe- 
rience of inhabiting as encompassing two distinct dimensions: firstly, the 
physical, ontological, and phenomenal aspects of existence; secondly, its 
extension into the digitized realm of e-society, where AI plays a vital role. 
Within this spectrum, a hybrid, pulsating, and creative world emerges. 
Semiotics, the study of signs and the establishment of fluid catego- 
ries, becomes instrumental in navigating the relationship and transition 
between these two realms. It provides a foundation for the processes of 
reading, writing, translation, and transduction that occur as signification 
unfolds across intersecting spaces. The incorporation and unbundling of 
physical-algorithmic magnitudes, both individually and collectively, shape 
this dynamic interplay. The embedded face reflects the material substance 
it conveys while the virtual face speaks to a temporary dematerialization 
destined to exist within the ephemeral realm of binary language, occasion- 
ally slipping into oblivion. Digital images of faces persist, multiply, and 
are occasionally forgotten, albeit fleetingly. The associated data, however, 
remain a tangible and enduring element. Thus, the physical-algorithmic 
extension, influenced by AI, captures our attention due to its heterogene- 
ous and multifaceted nature. The chapter seeks to comprehend the various 
gradients, intensities, and varieties of physical-algorithmic bodies, their ori- 
gins and development, the forms of corporeality they embrace or exclude, 
and, most importantly, the spaces where this syncretism occurs. Barbotto 
explores how they integrate into the realms of reality and the digital sphere 
and how their habitability is constituted. 

In the eleventh chapter, Elsa Soro deals with social networking services 
(SNS) as the primary platforms for the exhibition and portrayal of digital 
faces. The expansion of the digital face space has been significantly accel- 
erated by the global pandemic caused by SARS-CoV-2, which has fun- 
damentally reshaped the public spaces of social interactions and the pri- 
vate sphere of affectivity on a global scale. While social media platforms 
like Facebook and Instagram have been extensively examined for their 
impact on physical interactions and their role in shaping new aesthetic 
norms, dating sites, and applications have not received the same level 
of scrutiny. To address this gap, the chapter aims to explore the facial 
space generated by dating sites, specifically examining the “inter-face” 
of these platforms as a space that facilitates the reimagining and reenact- 
ment of intimate face-to-face interactions. The study does not intend to 
focus on a particular dating app or site. Instead, its objective is to com- 
prehend online intimacy as a discourse constructed within the polyphonic 
landscape of various dating platforms and to contextualize it within the 
broader facialization of experiences facilitated by neoliberal models of 
digital platforms. 


xviii Preface 


In the twelfth chapter, Federico Bellentani deals with digital memori- 
als, which appeared in the late 1990s and developed since then due to the 
growing availability of digital technologies and the Internet. Commemora- 
tions and mourning have since diversified significantly in different digital 
memorial technologies: memorial websites, tribute pages, social network- 
ing sites, blogs, and virtual commemorations on video games. Faces are 
central elements in digital memorials and commemorative practices, being 
it both inter-faces providing access to the memories of the dead and texts 
constantly invested by multifaceted interpretations and practices. The face 
thus becomes a tool for commemoration and mourning, supporting users’ 
experiences during the time surrounding death. Exploring the connection 
between memorialization, face, and digital technology, the chapter inves- 
tigates digital memorials and commemorative practices and the mean- 
ings of faces within them. To do so, it first develops a semiotic-oriented 
framework for the design of digital memorials to ensure that digital tech- 
nology remains centered on human needs and meanings. It then goes on 
applying this framework by analyzing three kinds of digital memorials 
and commemorative practices, especially focusing on face-based ones: 
(i) institutional digital memorials; (ii) open, creative, and playful solutions 
to reimagine contested monuments and memorials; and (iii) institutional 
and vernacular digital-native commemorative practices. 

Massimo Leone 


Note 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 
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1 Masked faces 


A tale of functional redeployment 
between biology and material 
culture! 


Marco Viola 


1. The face 


What is a face? Why does it have the shape it has in our species? Addressing 
such questions will require a detour across several disciplines. But before 
interrogating scientific disciplines, each coming with its own lexicons and 
ontologies, it seems safe to anchor our inquiry to a term coming from ordi- 
nary language. 


1.1. The face as a seat of functional organs 


According to the Cambridge Dictionary online, the first definition of “face” 
is “the front of the head, where the eyes, nose, and mouth are located”.* 
Indeed, the human face hosts three of the five senses: taste, smell, and sight. 
Ears (for hearing) are not far. As for the skin (for touch), while it is spread 
all over the body, the density of mechanoreceptors (hence the sensory accu- 
racy) of facial skin may be even superior to that of the fingertips (Con- 
nor and Abbs 1998). Thus, the honorary label of “sensory headquarter”, 
which Adam Wilkins repeatedly employs in his monumental book on the 
evolution of the human face (2017), seems particularly fitting for the face. 

Yet this is not the whole story. In fact, nasal and oral cavities are gates 
not only for sensory inputs but also for vital substances, that is, oxygen 
and nutrients. In sum, while relatively small compared to body size (at least 
in adult humans), the face hosts a lot of biologically relevant organs. To 
Wilkins, one reason for this functional overcrowding is that “it presumably 
is somewhat advantageous when seeking food to have sites of the three 
of the major senses grouped near the mouth and not far from the brain” 
(2017: 358). However, having a face is not a uniquely human asset. Nor 
is unique to primates. And even extending to all mammals is not enough. 
Actually, 


[iJn all ages and in all branches of the animal kingdom a face of some 
sort has been indispensable to all but sessile animals, just because a 
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face is concerned primarily with: The detection of desirable sources of 

energy; The direction of the locomotor machinery toward its goal; The 

capture and preliminary preparation of the energy-giving food. 
(Gregory 1929: 3) 


At this point, a reader might be asking, “What is the starting point of the 
evolutionary history of the face?” A hint toward the answer is afforded by 
the title of the book from whose opening I draw the aforementioned quote, 
Our face from fish to man (Figure 1.1). More precisely, to the best of our 
present-day knowledge, the earliest documented species that ever devel- 
oped a face (i.e., a face-like arrangement of sensory organs) was Entelo- 
gnathus primordialis (roughly “primordial complete-jaw”), a 419 million 
years old fish whose fossil was recently found in China (Zhu et al. 2013). 
But how come the classical prolonged faces of ancient fishes evolved to 
become the relatively flat faces distinctive of our species? The biologists 
who have addressed this question stress that this long evolutionary jour- 
ney was not shaped by a single factor but rather by a complex interplay 
of different selective pressures that paved the way to the emergence and 
transmission of different traits. To appreciate how it might be happened, 
an epistemological caveat is in order. Namely, that evolution does not pro- 
ceed exclusively through adaptation. An adaptation obtains when a trait 
T is selected because it enables or enhances a function X, where X-ing 
promotes the survival and reproduction of an organism that possesses T 
within its niche N. Sometimes, evolution also makes use of the so-called 
exaptation (Gould and Vrba 1982). An exaptation obtains when a trait T, 
originally adapted to perform a function X (or that played no function at 
all), is redeployed to produce or to enhance the production of an effect Y. 
Then, if Y-ing turns out to be particularly advantageous for an organism’s 
survival and/or reproduction within its niche, selective pressure may favor 
the individuals that possess exapted trait — a case which some scholars call 
secondary adaptation. A moral to be drawn here is that evolution is not 
linear. And neither is the relationship between traits and their functions. 
For the sake of the present discussion, the more relevant factors leading 
from the first mammals toward primate and eventually human facial shape 
were likely diet and brain expansion (Wilkins 2017; Lacruz et al. 2019). 
Differences in diet promoted adaptations of the jaws, indirectly affecting 
the mandible and the surrounding bones, whereas the increase in brain size 
had two main effects: firstly, the skull shifted forward to let more space to 
the braincase so that the to-be-human faces progressively acquired the verti- 
cal, almost flat arrangement distinctive of hominids (cf. the elongated muz- 
zle of most other animals, which mostly develop horizontally). Secondly, 
larger brains permit more sophisticated behavioral and sensory processing, 
including enhancements in vision and neural machinery specialized for face 
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1. Devonian shark; 2. Upper Devonian air-breathing, lobe-finned fish; 
8. Lower Carboniferous amphibian; 4. Permo-Carhoniferons reptile; 5. Triassic 
mammal-like reptile; 6. Cretaceous mammal; 7. Lemuroid primate; 8. Recent 
Old World monkey: 9. Chimpanzee: 10. Tasmanian; 11. Roman athlete. 


Figure 1.1 The illustration on the frontispiece of Our face from fish to man (Greg- 
ory 1929), summarizing the development of face throughout evolution. 
Freely available at www.biodiversitylibrary.org/item/41026. Copyright: 
No known copyright restrictions as determined by scanning institution. 
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perception. Compared to most other vertebrates, primate vision comes with 
three perks: (i) contrary to animals with more lateralized eyes (e.g., fishes), 
the visual fields of primates, whose eyes stand close and almost in paral- 
lel, overlap up to 120°. Evolutionary speaking, primates ‘bargained’ the 
amplitude of the visual field (fishes and most quadrupeds can see a wider 
portion of their surroundings) in favor of stereopsis, that is, the ability to 
estimate depth due to binocular vision. (ii) As larger brains permit more 
cortex dedicated to vision, primates develop a more acute vision for near 
objects. (iii) The structure and function of photoreceptors in primates’ reti- 
nae allow for a good trichromatic vision (Wilkins 2017: 226-227). While 
the first evolutionary advantages fostering primates’ vision were probably 
related to increased efficiency in gathering food and avoiding predators, 
it is reasonable to assume that such a perceptual enhancement allowed 
the exaptation of faces for social functions, enabling the fine-grained dis- 
crimination of different individuals within the same species and allowing to 
extract cues about these individuals’ likely behavior. 


1.2. The face as a social organ 


The enhanced possibilities of social communication afforded by early 
hominids’ faces have likely proved so advantageous that they themselves 
became a relevant flywheel for survival and reproduction — hence becoming 
secondary adaptations as they provided a virtually effortless means to read 
off cues about a conspecific’s mental states or possible course of action. The 
final stage of this story is the Homo sapiens’ face whose basic shape we can 
see in the mirror. Among other things, (i) it has little (if any) fur covering it; 
(ii) it has its distinctive and attention-grabbing white sclera, a color we do 
not find in other primates (Kobayashi and Kohshima 2001); (iii) it hosts a 
highly sophisticated network of facial muscles, whose movements can pro- 
duce a host of facial expressions, not to mention (iv) lip muscles, regulat- 
ing the articulation of spoken language. While the ability to perceive faces 
seems widespread in the animal kingdom, not only among mammals but 
even in distant species such as corvids (see Leopold and Rhodes [2010] for 
a beautiful review), human brains possess neural regions supporting very 
nuanced face processing within occipital and temporal lobes (Kanwisher et 
al. 1997; Grill-Spector et al. 2017).3 This second, socially driven stage of 
the evolution of the face (and of a brain specialized in perceiving it) leads 
Wilkins to claim that 


[flor the primates . . . and especially the anthropoid primates, and for 
the hominins most of all, the social face clearly has a unitary and inte- 
grated function — namely, that for sending and receiving signals (often 
complex ones in humans) about social behavior between individuals. 
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The whole face participates both in such signaling and as an identifier 
of the individual. 
(Wilkins 2017: 358) 


Indeed, human faces are a powerful tool for communicating a vast amount 
of socially relevant information (Jack and Schyns 2015). When they are 
presented in our visual field (or even just subliminally perceived at its 
periphery), faces grab our attention (as well as the center of our visual 
field, thus enabling the more accurate foveal vision) and automatically trig- 
ger our decoding of social information conveyed by several cues (Palermo 
and Rhodes 2007). Some information is conveyed both by static aspects 
of the face such as shape and complexion. Other information is conveyed 
by dynamic cues, namely the movements of the eyes (revealing where the 
gaze is directed) and facial muscles. Given the number of muscular groups 
we can independently move, Lee and Anderson (2017: 175) estimate that 
by combining them, we can produce up to 3.7 x 101° different expressions. 
Indeed, the second definition of ‘face’ to be found in the Cambridge Online 
Dictionary, “an expression on someone’s face”, alludes to this highly com- 
municative function played by facial muscles dynamics. 

The idea that faces convey socially relevant information by redeploy- 
ing movements originally selected for other purposes was authoritatively 
proposed by Darwin in his pioneering book The expression of emotion in 
man and animals (Darwin 1872). Darwin aimed at contrasting Charles 
Bell’s creationist explanation of the link between human facial muscles 
and emotional expressions. A profound connoisseur of facial muscles, 
Charles Bell (1824) held that such complex musculature must be a gift 
God made to humankind to enable us to express our emotions. To defend 
evolutionism, Darwin had to find another explanation for human facial 
movements, based on evolutionary rather than on creationist grounds. 
Supported by several observations, which he either made himself or col- 
lected from various sources, Darwin stresses how several patterns of facial 
(and bodily) movements related to emotional episodes are similar not only 
in humans of disparate cultures but also in several mammals. To explain 
these commonalities, he appealed to three principles: the principle of ser- 
viceable habits, that of antithesis, and that of the direct action of the nerv- 
ous system. For the sake of brevity, I will only discuss the former, which is 
also the most relevant for the present discussion: the principle of service- 
able habits. 

To put it roughly, the principle of serviceable habit asserts that emo- 
tional expressions are a vestige of some movement that was useful in some 
phylogenetically old species, even if it does not serve any direct function 
anymore. Let me clarify with an example. In the chapter dedicated to 
expressions of hatred and anger, Darwin reports how, when human beings 
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across the world undergo an anger outburst, “the shape of the[ir] mouth 
wholly changes, and the teeth are exposed” (1872: ch. 10). He goes on not- 
ing how “[t]his retraction of the lips and uncovering of the teeth during par- 
oxysms of rage, as if to bite the offender, is so remarkable, considering how 
seldom the teeth are used by men in fighting” (ibidem). Why then exposing 
the teeth? His hypothesis is that “our semi-human progenitors uncovered 
their canine teeth when prepared for battle” (ibidem). Hence, since this 
habit was useful for our progenitors, it got transmitted to their descend- 
ants, namely humans and apes. And while it stopped playing its original 
function, that is, unsheathing the bite for an upcoming fight, this action has 
gained a social function, that is, expressing anger or menacing someone. 

The Darwinian idea that facial movements evolved twice — firstly to per- 
form some action, secondly to express some emotion and/or intention — is 
still quite popular nowadays (see, for instance, Shariff and Tracy 2011). 
Lee and Anderson (2017) dub the first class of survival-related behaviors 
played by facial movements egocentric functions and the second one allo- 
centric functions. For instance, they note how opposite sets of movements 
characterize fear and disgust expressions, the former opening and the lat- 
ter closing facial cavities. Their egocentric functions are hypothesized to 
help the defense against large threats such as predators by enhancing visual 
field (for fear) and against small threats like pathogens by closing cavi- 
ties (for disgust). But their allocentric functions are to inform conspecifics 
about these same threats. This communication does not need to be explicit 
or intentional. In effect, even when perceived unconsciously, facial move- 
ments seem capable of triggering facial mimicry, which is held to support 
emotional contagion and empathy, scaffolding social bonding in humans 
and in other primates (Palagi et al. 2020). 

Darwin was not the first scholar to propose hypotheses out on similari- 
ties between human and animal faces. Before him, several physiognomists 
across the centuries — from the Greek Polemon to the Arab al-Razi’s with 
his Firasa, to Renaissance thinkers like Giovanni Battista Della Porta up 
to Johann Kaspar Lavater in the XVIII — leveraged the similarity between 
the shapes of some humans’ face and a given animal to conclude that those 
humans possess the mental traits that (allegedly) characterize that animal. 
Yet these authors are nowadays considered eminently pseudoscientific.* 
A relevant difference worth stressing between them and Darwin is that the 
former aimed at inferring transient mental states such as emotions (based 
on dynamical cues such as facial movements). In contrast, physiognomists 
also inferred permanent or enduring mental states such as personality traits 
(usually from static features of the face). 

Pseudoscientific as it may sound, we do routinely infer personality traits 
and other information from facial first impressions, albeit very often sub- 
liminally. Like it or not, we are natural-born physiognomists: even a quick 
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look at a face suffice to form judgments about her bearer’s attractiveness, 
race, age, healthiness, trustworthiness, and competence — to name but a 
few qualities that we form a first impression based on the face.° And these 
judgments tend to be consistent among different observers, at least as long 
as they are from the same cultural group. Also, they are often ill-founded 
(Smortchkova 2022).° And yet they relevantly bias subsequent social inter- 
actions (Todorov 2017). 

Trivial as it may seem; another social information typically extracted 
from the face is its bearer’s identity. While all human (and most animal) 
faces present a similar T-shape configuration, when it comes to details, 
human faces present much intraspecific diversity. Hence, if we are suffi- 
ciently familiar with somebody’s face and we do not suffer from prosop- 
agnosia (a clinical condition characterized by the inability or very poor 
ability to recognize facial identities, first described by Bodamer [1947]), 
seeing somebody’s face automatically tells us who she is and prompts us to 
regulate our behavior toward her accordingly. 

Interestingly, psychologists have long held that reading identities and 
emotions out of faces relies on cognitive processes that are largely dis- 
tinct (Bruce and Young 1986), albeit scholars are still debating about how 
deeply this dissociation runs or whether it depends on different neural 
pathways (Haxby et al. 2000) or different information being diagnostic 
to identity and expression (Calder and Young 2005). This dissociation is 
highlighted by the fact that we seem still capable of ascribing emotional 
states to someone with above chance (but far from perfect) accuracy even 
if we are unable to detect their identity. Indeed, emotional expressions can 
be detected even by people suffering from prosopagnosia! 


1.3. Egocentric and allocentric functions of the human face 


Taking stock, in this section, I have discussed the double evolution of the 
face: firstly, as a collection of organs involved in (adopting Lee and Ander- 
son’s terminology) egocentric functions, comprising both sensory functions 
like seeing and survival functions like breathing. Secondly, as an attention- 
grabbing organ with an allocentric function, that is, for conveying socially 
useful information (e.g., about one’s identity or emotional state). A few 
remarks are in order before passing to the next section. Firstly, the abil- 
ity to extract socially relevant information from faces no longer per- 
tains uniquely to human beings (or animals, to a lesser extent). We build 
machines (often employing neural networks) that are getting pretty good at 
recognizing identity (Taskiran et al. 2020) or emotions (Huang et al. 2019; 
but see Tcherkassof and Dupré 2020). Secondly, the flow of social infor- 
mation from an observed face toward an observer’s eye does not always 
require deliberation from either side. On the one hand, people attribute 
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meaning to faces (and behave accordingly) regardless of their willingness 
or awareness. On the other hand, we cannot cease to possess the static 
features used as cues to infer our qualities or identity (although we may 
hide them, as discussed in the next section). And possibly, we neither have 
full control over our facial expressions of emotions.’ Unless we are faking 
it, of course. This brings us to another issue: we can use facial dynamics 
to deceive. By reproducing a given pattern of facial movements associated 
with a given emotion or mental state, we can trick an observer into think- 
ing that we are experiencing that emotion or entertaining that mental state. 
But more often, she tricks herself; that is, she makes wrong judgments 
about an observed subject’s mental state, social categories, or identity, 
based on some observed face. 

Thirdly, unlike egocentric functions, whose actual underlying structures 
are individual organs within the face, allocentric functions are realized 
more holistically. As Wilkins efficaciously puts it, “in its communicative 
aspects, the face is definitely more than the sum of its parts” (2017: 358, 
italics in the original). This is not to say that information can be extracted 
from a partial facial Gestalt — they can. More modestly, the social infor- 
mation conveyed by the face depends much more on spatial relations 
between individual elements of a face (and of their alterations due to mus- 
cular movements) than on specific details about these elements (e.g., the 
eyes being green or brown). In any case, if animal faces were already quite 
overcrowded with (organs implementing) egocentric functions, the emer- 
gence of allocentric functions exacerbates this functionally overcrowding. 
In the next section, we will see how this functional overcrowding affects 
the workings of a particular class of artifact that affects the face, namely 
the mask. 


2. The mask 


While in the previous section I have examined the face, offering an account 
(however sketchy) of its functions in the light of its evolutionary history, 
in this section I turn my attention toward the masks: tools whose functions 
are to act upon some function of the face. Thus, I will address questions 
like: what are masks? And what are they for? Again, let me start with ordi- 
nary language. The Cambridge Dictionary Online presents the mask as “a 
covering for all or part of the face that protects, hides, or decorates the per- 
son wearing it”. Such a definition may misalign with some of our linguistic 
practices. On the one hand, it includes objects that we typically would not 
properly call “masks”, such as sunglasses, makeup, or transparent face 
shields. On the other hand, it excludes some objects that some scholars 
call masks, such as the unwearable 90-foot-long Great Mask employed in 
religious ritual by the Dogon tribes (Grimes 1975) or the stimuli employed 
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in experimental psychology to cover a target stimulus in studies about 
unconscious perception. As this problem is unlikely to be solved simply by 
looking at another dictionary,’ for now I settle for this provisional working 
definition, which will be further qualified (and slightly modified) as I will 
sketch an ontology and taxonomy of masks. To do so, a detour in some 
philosophical theorizing on artifacts is in order. 


2.1. Artifacts and their functions 


To begin with, masks are a kind of physical objects, namely artifacts. Start- 
ing (at least) from Aristotle, Western thought distinguishes physical objects 
between natural and artificial, that is, respectively those that exist as such 
by nature and those whose existence and properties depend upon an act 
of crafting.’ 

Usually, but not necessarily, artificial objects are brought into existence 
by some author in order to play some function (or set of functions). This 
production typically entails modifying preexisting objects so that the final 
product acquires some intended properties, namely the properties that 
enable them to be used for a given function. For instance, early homi- 
nids produced rudimentary weapons by knapping stones to grant them the 
property of sharpness because sharpness enhanced their function: harming 
prey or enemies. When this process of crafting succeeds, the resulting prod- 
uct is an artifact (Hilpinen 1992). 

It is worth mentioning that not all objects that exist by virtue of some- 
one’s action are artifacts: indeed, some may be unintentional products or 
by-products of something else — in short, scarps. Moreover, and more rel- 
evantly for the present discussion, not all the properties of an artifact are 
intended properties. Indeed, some of its properties are unrelated to the pur- 
pose it was created for — let me dub them unintended properties. To recap 
some salient points, artifacts can be defined as (i) physical objects that (ii) 
have been created by some author so that (iii) they embed some properties 
that (iv) allow them to play a given function (or a set of functions). Thus, 
the intended function of an artifact works as a sort of teleological attrac- 
tor that, provided that the crafter has the right skills and materials, leads 
toward a certain endpoint of the craft. Functions are so central in artifacts 
that some of them are named after the function they play — think about 
potato-peelers, screwdrivers, and can openers. Moreover, when we need 
to categorize artifacts — either for intellectual purposes or for more mun- 
dane scopes, like organizing the shelves of a supermarket — we place much 
more weight on their functions rather than on non-functional properties 
like their color. Yet the relation between artifacts and functions is all but 
straightforward. As highlighted by Preston (2009), artifact functions are 
multiply realizable; that is, they can be performed by artifacts of several 
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shapes and materials, provided that some basic constraints are met. On 
the other hand, artifacts are often multiply utilizable (or multifunctional): 


[T]ypically they are designed to serve several [functions], often simultane- 
ously. Uniforms, for instance, serve the functions of ordinary clothing -— 
keeping the wearer warm, dry and modest — but they also serve 
identification functions with regard to telling friends apart from enemies, 
displaying rank, and so on. 

(Preston 2009: 215) 


An interesting aspect of multiple usability is that artifacts may be employed 
to subserve functions they were not produced for. For instance, imagine that 
you receive an urgent Zoom call while reading these very words (assum- 
ing that you are reading them on a physical copy of the book). Your table 
is slightly too low so that your face does not fit in the camera embedded 
in your laptop. So you take this very book and put it under your laptop. 
Admittedly, this was not the function that the publisher or the authors of 
this volume had in mind when they worked to produce the book. Notice 
that “[t]hese alternate functions sometimes become standardized or rou- 
tine” (ibidem), either for a single individual or for a collectivity. The chair 
in my bedroom, originally designed to host my buttocks, is more often 
employed to host some of my clothes. 

Sometimes, alternate functions become so prominent that they take over 
the privileged status of intended functions when further tokens of that 
given artifact are produced. The history of technology abounds with such 
cases. A curious example from pharmaceutics concerns the molecule Silde- 
nafil. Initially created by Pfizer to cure angina pectoris, the drug proved 
inefficacious with respect to its intended function. However, it turned out 
to have an unexpected side effect in fighting erectile dysfunction. When it 
got commercialized as Viagra, it became a source of much income (and 
charges of disease-mongering) for Pfizer and a model case of drug reposi- 
tioning for the chemical industry (Ashburn and Thor 2004). Since the func- 
tional properties that subserved the once-intended function may misalign 
with those that best subserved the new-intended functions, new versions of 
that artifact are modified to privilege this newly interesting set of functions, 
even at the expense of others. When the function(s) of the new artifact dif- 
fers sufficiently from that of its ‘progenitors’, we tend to perceive the new 
artifact as not only a new token of the same artifact type but a different 
type of artifact altogether. If this story sounds familiar, it is because, in 
many respects, it reflects the dual biological evolution of the face presented 
in the previous section. 

Before coming back to our topic — the mask — I want to emphasize 
another feature of artifacts that has received relatively little attention thus 
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far: their unintended effects. When I wake up in the middle of the night 
and walk through the bathroom, I might stumble upon my chair and fall 
on the floor. Unintended effects are best kept distinct from alternate func- 
tions because they are not the product of anyone’s intention. Arguably, my 
fall is not due to some nasty pixie’s intention to exploit the chair to make 
me fall. It occurs independently from anyone’s will in virtue of some of the 
properties of the chair (e.g., it does not glow at night, it stands between the 
bathroom and my bed, and so on). As we will see, in designing artifacts, 
avoiding the properties yielding undesirable unintended effects can be as 
important as implementing the intended properties. 


2.2. Masks as artifacts 


This brief detour in the philosophy of artifacts affords a more nuanced 
answer to the question “what are masks?”. Masks are artifacts whose 
function is to interact with the functions of the face. To stay as close as 
possible to linguistic intuitions, this definition needs to be enriched by two 
additional constraints: (i) covering condition: masks ought to be placed 
upon (at least some part of) the face. As such, they must be physical 
objects, or at least they need to be able to interact with at least one sensory 
modality (a hologram can still count as mask). (ii) reversibility condition: 
donning a mask must be easily reversible. While makeups can be washed 
easily, removing ink tattoos is far harder; only the former shall count 
as mask. Recall from the previous section that the face (or the organs it 
hosts) has been associated with two classes of functions: egocentric func- 
tions, that is, sensory and vital functions; and allocentric functions, that 
is (voluntary or involuntary) broadcasting of social information to other 
living beings. Building on this distinction, we can sort masks into two 
broad categories, depending on which functions of the face they are meant 
to act upon. 

On the one hand, we have what I shall dub protective masks. They can 
be distinguished into two sub-classes. Firstly, we have bodily-generic pro- 
tective masks, that is, artifacts built to protect the face qua bodily part, 
preventing or mitigating physical harm or other environmental hazards 
(e.g., cold) that would equally affect any part of the body. Secondly, we 
have face-specific protective masks, that is, artifacts that specifically act 
upon some of the exchanges of sensory information (e.g., light) or vital 
substances (e.g., oxygen) occurring between a subject and the environ- 
ment, and that are gated by face organs. The former category might include 
hockey masks and balaclavas (ski masks). The second might include anti- 
gas or surgical masks, as well as artifacts that we would not commonly 
call masks, such as nose filters (protecting from some particles in the air) 
and sunglasses (protecting from the light); or even blindfolds (preventing 
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from sight) — after all, nobody told that protective mask must be in the best 
interest of the wearer! 

On the other hand, we have what I shall dub social masks: artifacts built 
to modify some function of the face qua social organ. This includes numer- 
ous religious and secular masks, ranging from those employed by ancient 
religious ceremonies to those employed in contemporary swinger clubs (for 
a comprehensive review, see Inglis 2018). Moreover, other objects befall in 
this category despite probably not counting as mask for ordinary language: 
for instance, various sorts of hijabs, the veils employed by Muslim women 
(Leone 2010), or various sorts of make-ups, such as the emotional make- 
up employed in theater (Magli 2013, §4.8).1° 

To laypeople, social masks are arguably more prototypical than protec- 
tive masks. Whereas to many social scientists, only social masks count as 
masks in a proper sense (cf. Lynteris’ quote in §3). This may be because, 
for many of anthropology and semioticians, the real phenomenon of inter- 
est is not the mask per se, but rather the function (or better, set of func- 
tions; see later) afforded by social masks, that we can call masking: altering 
someone’s display of identity or mental states (Pollock 1995). The focus of 
interest of these thinkers is not totally aligned with that of the present dis- 
cussion, because there exist masks whose intended function is not masking 
(namely, protective masks), and because masking is possible even without 
a mask (recall Preston on the multi-realizability of functions). For instance, 
Grimes (1975) construes dance as a means for embodied masking. 

Yet, as anticipated, ‘masking’ is best understood not as a single function, 
but rather as a set of functions: (i) concealing some properties that carry 
social information; (ii) altering them; and finally (iii) the addition of new 
properties that afford further social information.'! Admittedly, as soon 
as we move from theory to practice, the conceptual distinction between 
altering and adding immediately becomes blurry. Nonetheless, I think it is 
worth preserving it at least in principle, also by making the following quali- 
fication: while alteration modifies the social information conveyed by the 
face by modifying some of its existing traits (e.g., by enhancing or reducing 
the saliency of some organs or by modifying skin pigmentation), addition 
adds new social information by adding features that the original face did 
not possess (e.g., fox-shaped Venice Carnival masks evokes cunning with- 
out appealing to any feature of the wearer’s face). 

Moreover, each of these three functions can differ with respect to the 
specific social information it targets: for instance, if you plan to go to a 
swinger club, you might want both to don a mask on your eyes to con- 
ceal your identity and perhaps to use an anti-age make-up to conceal your 
wrinkles. Both are concealing masks, although they are meant to hide dif- 
ferent aspects of social information. The combinations between the three 
masking functions and specific social information targeted is constrained 
by several considerations. Most prominently, it is worth noting how the 


Masked faces 13 


rigidity that characterizes most masks disallows the possibility to add any- 
thing to expressions, which would require some technological analog of 
facial muscle dynamics. In fact, neutral masks (white and expressionless — 
cf. Beato 2022) were introduced by LeCoq’s school to train actors, so that 
they learn to “de-centralize” their expression to the remainder of the body 
since the body is arguably easier to see than the face in the last row of a 
theater. Of course, a mask can represent some expression, but only stati- 
cally and abstractly, possibly engendering a feeling of uncanny and disal- 
lowing facial mimicry. Hence, arguably a mask can be used to conceal 
one’s expression, but not to deceive about it. 

Thus, additive masks typically target identity. The reason why masks 
seem more prominent for playing social roles of identification than other 
clothing artifacts targeting different part of the bodies is likely that faces has 
historically been, and will likely be in the future,! the more relevant source 
of information about one’s identity. This explains why wearing masks in 
public without reason is illegal in many countries. In most cases, whoever we 
meet, we first look at their face. And even when a mask covers the attention- 
grabbing features of the face, for example, the white sclera, it capitalizes on 
our habit, whenever we meet someone, to look at her face first. From reli- 
gious rituals to contemporary cosplaying, many masks are produced with 
the aim of letting the wearer assume the identity of someone — a specific 
person or character — or something — a generic social role or natural entity. 

While hyper-realistic masks, which proved capable of fooling people 
most times (Sanders et al. 2017), do add a layer of identity, their intended 
function is likely that of concealing: adding someone else’s identity is just 
because displaying no identity at all (as it would happen wearing a bala- 
clava) would raise way too many suspicions (cf. Gramigna 2022a). Quite 
on the contrary, those masks that are produced to add a layer of identity do 
not explicitly seek to conceal the wearer’s identity but only so to speak to 
temporarily “suspend it”. It should be noted that several anthropological 
reports suggest that, in some cases, the surplus identity afforded by masks 
does not only affect the behavior of the observers: it somehow seems to 
‘sink’ into wearers’ own mind, affecting their phenomenology and behav- 
ior (Honigmann 1977)." 

Interestingly, some masks may serve the purpose of granting anonym- 
ity not in virtue of their concealing function so much as by means of their 
additive function, that is, by positively representing the desire to stay anon- 
ymous. For instance, in describing the rising popularities of vizard masks in 
eighteenth-century London, Heyl (2001: 119) notes that 


it was still relatively easy to recognize the wearer of sucha mask which... 
just covered the upper half of the face. But even this must have been 
more than sufficient to introduce the idea of anonymity and therefore to 
modify the behaviour of the wearer. 
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Examples of altering functions of the masks include the amplification and 
distortion of voices, which was common in theater masks (Twycross and 
Carpenter 2017: 8), or the aforementioned uses of makeups to alter per- 
ceived age, or its use in theatre actors to emphasize certain features of the 
face expressing emotions. However, I find it hard to conceive identity- 
altering masks because identity, unlike expression or other facial features, 
does not admit degrees or variations: you are either someone (or some- 
thing) or not. 

Figure 1.2 provides a schematic summary of the conceptual articulation 
of masks’ functions. As anticipated, like most conceptual boundaries, they 
often blur when applied to real world. This is partly due to the aforemen- 
tioned property of artifacts to be often multi-usable, be it due to the authors 
themselves producing the artifact with multiple functions in mind, be it due 
to the users’ choice to exploit them in unexpected ways. Moreover, due 
to some unintended effects, artifacts can have effects that transcend both 
the author’s and the user’s intentions. But when it comes to the mask, the 
implications of multi-usability and unintended effects are complicated by a 
further fact: namely, that the face is functionally overcrowded, as discussed 
at the end of §1. Taken jointly, these features bring about several interest- 
ing consequences, some of which will be dealt with in the next subsection. 


2.3. Redeploying mask 


In the popular Spanish TV series Money heist (La casa de Papel), a group 
of robbers coordinated by the brilliant mastermind known as “The Profes- 
sor” penetrates the state mint in Madrid to print billions of euros. The rob- 
bers could have been easily killed by police snipers, were not for a clever 
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Figure 1.2 A schematic representation of the taxonomy of masks based on their 
(intended) function, depending on how they interact with the functions 
of the face. Copyright: Image created by the author. 
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stratagem: they took hostages and get them wearing the very same face 
mask wore by the Professor’s team, representing the iconic face of Salvador 
Dali — along with a red jumpsuit and occasionally fake weapons. By so 
doing, they prevented the snipers from shooting: they cannot gamble when 
the hostage lives are at stake! However, at some point the policewoman 
in charge of arresting the robbers decided to exploit their same strategy 
of anonymity: a rescue squad wearing the same masks and suits was sent 
infiltrating into the mint through a secret tunnel. Yet pretty soon the police 
realizes that the Professor predicted and countered this move by having 
everybody (hostages and robbers) switching the Dali mask with another 
representing the Scream of Munch. The infiltrators thus had to abort the 
mission. In fact, if they entered the mint with the ‘old’ mask, they would 
have been easily recognized and captured. In this implausible case, while 
the author that produced those masks has likely conceived them as amus- 
ing identity-adding artifacts, the identity-concealing or the identity-adding 
function of the mask was equally crucial for the Professor when he elabo- 
rated the plan. 

Real-life cases can be even more interesting for the present discussion. 
In actual heists, rather than a mask recalling some artist’s face, robbers are 
more likely to wear impersonal black balaclavas to hide their facial fea- 
tures. Obviously, balaclavas are not originally produced or commercialized 
to conceal identities: instead, they are intended as a protection of the facial 
region from cold temperature. A similar redeployment of a protective mask 
seems to have occurred with vizard masks in XVIII England: “[t]he main 
practical reason for wearing these winter masks . . . seem to have been the 
desire to protect the delicate skin of the face from the cold” (Heyl 2001: 
118-119). However, as people were beginning to feel the urge for privacy, 
the ability of these masks to warrant anonymity (or at least to signal the 
desire to be anonymous) became prominent: 


[T]he more utilitarian aspects of wearing a mask became a mere pretext. 
What had been the mask’s side-effect (the price a woman had to pay for 
“guarding her complexion” was that it temporarily rendered at least 
part of her face invisible) now became its main attraction. 


(Ibidem) 


Now consider sunglasses. While the first proto-sunglasses might have been 
originally conceived to protect from excessive light, nowadays they are 
often worn for shielding one’s spontaneous emotional expression from 
the onlookers’ gaze, as well as for avoiding social-inviting eye contact 
(Viola 2022). Moreover, the history of sunglasses throughout the twenti- 
eth century shows that they have been imbued by social meaning of cool- 
ness (Brown 2015). Sometimes, the social meanings of facial adornments 
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are subverted for the sake of political appropriation, as in the case of gas 
masks becoming a symbol of rave culture (Voto and Soro 2022). 

It is worth noting that masks are not the only kind of artifact play- 
ing both protective and social functions. Recall that, in introducing mul- 
tiple usability, Preston discussed the case of a uniform (§2.1). Later on, 
she points out that “the example of uniforms illustrates [that] multiple 
functionality is often a matter of serving social or communicative func- 
tions in addition to purely technical ones” (Preston 2009: 215). The point 
might probably be pushed further by claiming that any cloth, and possibly 
any artifact, unwittingly or not, always adds some sort of social informa- 
tion, whose decoding can be affected by cultural and idiosyncratic factors 
(Leone 2010). Consider balaclavas: their association with criminals has 
become so iconic that, unless we find ourselves in a very cold environment, 
merely seeing a person wearing one balaclava we would likely think that 
he or she is a robber. 

Yet the case of masks presents further peculiarities because faces are 
functionally overcrowded. Since various important vital and social pro- 
cesses occur on the human face, choosing the best materials and shape for 
conferring the intended properties to a mask is only half of the job for a 
good mask crafter (or mask designer). The other half consists in avoiding 
unintended properties that would interfere with other functions of the face. 
It does not matter if a brand-new model of balaclava does a perfect job in 
preserving bodily heat, if this comes at the expense of not letting holes in 
the eyes, or if its material prevents breathing. Possibly, it would be equally 
hard to sell sunglasses that protect from the light but that make the wearer 
look ugly. Trivial as it may sound, this issue has a host of implications that 
should be taken extensively into account by mask designers. 

Building on all these premises, in the next and final section I will try to 
see how these considerations fit with a specific case study, namely that of 
medical face masks. 


3. Some remarks for a case study: the medical face mask 


In 2020, the COVID-19 pandemic changed the face of the world as well as 
the look of faces around us. Indeed, as many governments and institutions 
encouraged or enforced the use of medical face masks in public spaces, 
they became almost ubiquitous. From the point of view of the taxonomy 
sketched earlier, medical face masks are face-specific protective masks: they 
are produced to interact with the wearer’s breathing, filtering droplets that 
may carry the infection. In a sense, they might be thought of as an artefac- 
tual analog of disgust. Traditionally, disgust is conceived as “a guardian 
of the temple of the body, responding to direct threats of contagion or 
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infection to parts other than the mouth” (Rozin et al. 2008: 764). Just like 
any good guardian shuts the doors of the temple if she senses a threat, spot- 
ting a possible pathogen triggers a contraction of facial cavities (mouth, 
nostrils, and eyes), shaping the face into the emblematic expression of dis- 
gust (Lee and Anderson 2017). As it helps the immune system by prevent- 
ing infections, disgust is held to play an important role in the so-called 
behavioral immune system (Schaller and Park 2011). Yet useful as they 
may have been in some evolutionary tasks, our innate emotions are insuffi- 
cient to cope with all possible challenges that the world confronts us with — 
especially when invisible pathogens are involved. Fortunately, culture and 
technology can help us in dealing with more complex threats. Thus, as 
the role that face masks play in avoiding infection presents some sugges- 
tive parallelism with that played by disgust, I propose that they (along 
with other similar measures) are part of our technological immune system. 
An interesting difference is that, while disgust protects oneself, most face 
masks are predominantly meant to protect the community from the patho- 
gens that the wearer may spread. 

But given the relevance of the social functions of the face, face masks 
turned out to have a lot of unintended social effects too. Since 2020, 
experimental psychologists have delved deeper into the exploration of such 
effects. In the slogan of the psychologist Erez Freud, “We are participating 
in the biggest experiment in human history in terms of face perception. All 
of us are wearing masks all day long, and we need to recognize different 
people around us”. A plethora of psychological experiments have now 
established that face masks interfere with identity and emotion recognition 
(for a review, see Pavlova and Sokolov 2021). By doing so, they also inter- 
fere with emotional mimicry, thus possibly dampening empathy and social 
bonding (Kastendieck et al. 2022). 

Moreover, face masks impair verbal communication, both because 
they impede lipreading, which implicitly helps in decoding oral language 
(McGurk and MacDonald 1976), and because they dampen some acoustic 
signal, depending on its frequency and on the type of mask (Corey et al. 
2020; Magee et al. 2020). The larger discomfort, however, was reported by 
people with hearing impairments due to their greater reliance on lipread- 
ing (Saunders et al. 2020). Not to mention that mouth movements are part 
of many sign languages. This is why many masks were invented that are 
totally or partially made of plastic materials, so as to uncover the face or at 
least the mouth region. While some such masks were already available even 
before the pandemic (e.g., the ClearMask™), many more were designed in 
2020, either by firms or by single sewers. 

Transparent masks for the hearing-impaired provide some food for 
thought for philosophers of technology. A variant of traditional masks, 
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transparent masks pose the challenge of preserving the intended protec- 
tive function, while at the same time eschewing (or minimizing) the unin- 
tended effect of concealing verbal communication based on lipreading and 
sign language. In facing this challenge, given that the face is functionally 
overcrowded, designers must pay attention not to interfere with other 
functions. For instance, while the prototype depicted in Figure 1.3 was 
particularly efficacious in its role as an artificial immune system by vir- 
tue of a very thick filter, this very filter had the side effect of hampering 
the passage of oxygen, making it unserviceable! At the same time, one 
unintended but potentially positive effect of transparent masks is that they 
restore the readability of emotional expression: as we have shown (Marini 
et al. 2021), face masks with transparent windows on the mouth restore 
the possibility to correctly judge the emotional expression on a face. 
Moreover, while experimental psychology has cleverly documented how 
face masks subtract social information (concealing), other social sciences 
are more suited to investigate how face masks also add a layer of symbolic 
meaning (adding). The medical anthropologist Lynteris documented it with 


Figure 1.3 A picture of myself wearing a prototype of transparent face mask taken 
in late 2020. The prototype that never made it to industrial production. 
Copyright: Image created by the author. 
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respect to the use of medical face masks to contrast the 1910-1911 plague 
outbreak in Manchuria: 


[W]hile being a practical, and in some cases effective, prophylactic 
technology, its material application has always already been tied to a 
symbolic function [that] renders these devices into masks, in the proper 
sense of the term [i.e. social masks]. 

(Lynteris 2018: 448, italics in the original) 


Similarly, at the onset of the pandemics Leone (2020d: 47) noted that “as 
the medical face mask becomes the most diffused facial device throughout 
the entire planet, even more than glasses, it ceases to be a mere functional 
object and acquires a more and more complex anthropological and semi- 
otic status”. As he correctly stresses, the meaning of face masks is modu- 
lated by cultural and historical coordinates: while more eagerly accepted 
in Eastern countries (possibly also due to the experience of the first SARS 
outbreak), initially they met resistance in Western countries, possibly due 
to a “deep-seated semiotic ideology, according to which human beings 
must show their face bare, with their mouth and nose and eyes visible” 
(Leone 2020d: 53). However, this situation evolved in the following years. 
Psychological evidence suggests that even in Western countries face masks 
may be associated with an increase in trustworthiness, possibly because 
they are associated with compliance with social norms and care for others 
(Olivera-La Rosa et al. 2020). But even within the same cultural group, this 
meaning is not immutable. Interviewing a sample of Chinese Hong Kong 
inhabitants, Siu (2016) finds out how wearing face masks was taken as a 
positive social norm during the SARS outbreak, whereas after the outbreak 
mask-wearers were stigmatized. 

In Western countries, wearing face masks — or ostensibly omitting to 
wear it contravening social norms — was sometimes imbued with political 
meaning (Inglis and Almila 2020). While in pre-COVID era not wearing 
a mask was simply the normality, during the 2021 lockdown it did signal 
either some disbelief about the received wisdom about the virus or a politi- 
cal stance of boycotting the sanitary measurements introduced or endorsed 
by some political group (and often both). Finally, just like other masks, we 
should expect that face masks somehow affect the wearer’s sense of self 
and behavior. Due to space constraints and limited evidence, documenting 
these effects must be postponed to another day. 

In sum, as they cover the most functionally overcrowded surface of the 
human body, namely the face, face masks are doomed to play some social 
role regardless of their intended use. Given that nowadays, after about 
three years from the onset of the pandemic, face masks appear to have 
become a relatively stable and ‘normal’ piece of clothing, social scientists 
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should join forces in order to fully comprehend their complex, contextually 
variable and multilayered set of social functions and unintended effects. 
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This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). I am grateful to several people for their encouragement and advice, 
which helped me to ameliorate the paper: Marta Calbi, Fabrizio Calzavarini, 
Marco Fasoli, Alex Machin, Gabriele Marino, Agostino Pinna Pintor. 
https://dictionary.cambridge.org/dictionary/english/face 

While no scholar denies that some brain areas, and especially the so-called 
fusiform face area (FFA; Kanwisher et al. 1997), have a crucial role in process- 
ing faces, the exact etiology of this specialization is currently under debate. On 
the one hand, some claim that the FFA is involved in the holistic recognition 
of several stimuli categories, provided that the observer has sufficient expertise 
with it, and consider that all sighted humans perceive many faces. On the other 
hand, evidence about face-specificity of FFA in blind people and newborns 
seems to testify that ontogeny may not be necessary. 

See Khalidi and Khalidi (2018) for some interesting epistemological considera- 
tions about the (pseudo)scientific status of Arab al-Razi’s Firasa. 

The belief that we may extract several information from others’ face has met 
alternate fortunes across centuries and cultures. For instance, in XVII England 
“i]t was a common notion that looking closely at someone’s face enabled one 
to ‘read’ this face and thus to find out about a person’s abilities, inclinations 
and motivation” (Heyl 2001: 121). Different cultures, however, regard differ- 
ent parts of the face as more or less expressive. Western thought conceived the 
eyes as a mirror of the soul at least since Aristotle (Synnott 1989), whereas “[f] 
or Kwakiutl, it was the mouth that gave access to the soul and which served 
as the link between the exterior identity and the interior soul” (Pollock 1995: 
587-588). 

It has been proposed that some personality judgments may depend on a process 
of overgeneralization: when some face’s shape slightly resembles some specific 
facial quality (e.g., a face with hairy brows recalls the typical anger expression, 
characterized by furrowed brows), that very quality gets reified as if it pertains 
to the essence of the face-bearer (see Zebrowitz 2017). 

See Glazer (2019) for an interesting proposal concerning the automaticity/de- 
liberateness of facial expression. 

The meaning of the terms of the natural language and the underlying concep- 
tual schemes are whimsical, and their evolution unpredictable, resulting from 
the random encounters of several contingencies. In a slogan, they are a second 
Wittgenstein affair, and no first Wittgenstein formal definition can ever hope to 
capture them. 

The boundaries between natural and artificial objects may be sloppy as well as 
deceitful. For instance, while dogs and seedless grapes may strike as intuitively 
as natural, they may well qualify as biological artifacts as their existence and 
characteristics depend upon human intervention (see Sperber 2007). 

Magli (2013, §1.3) is skeptical about subsuming makeup within the category 
of masks because the latter are traditionally taken to hide the face, the former 
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to modify it, for example, by enhancing the salience of some of its traits. This 
concern shall not bother us inasmuch as the notion of mask stipulated in the 
present discussion does not require hiding the face as a necessary condition. 

Is addition distinct from alteration? While the distinction is blurry, I think it is 
worth preserving it, also by making the following qualification: while alteration 
modifies the social information conveyed by the face by modifying some of its 
existing traits (e.g., by enhancing or reducing the saliency of some organs or by 
modifying skin pigmentation), addition adds new social information by adding 
features that the original face did not possess (e.g., fox-shaped Venice Carnival 
masks evoke cunning without appealing to any feature of the wearer’s face). 
Interestingly, the diffusion of face recognition technology spurred the develop- 
ment of ingenious social masks meant to conceal the wearer’s identity not from 
human eyes but rather from cameras (Biggio and Dos Santos 2020). 

Does the capacity of masks to affect the wearers themselves depends merely on 
their beliefs and self-suggestion and on the actual or conventional anonymity 
afforded by the mask? Or does it also depend on congruent responses of the 
other people accepting this pretense plays? The issue is still unsettled. Despite 
some suggestive findings, for example, that some children ceased to stutter 
while wearing a mask (Pollaczek and Homefield 1954), psychological investi- 
gations on the wearer-transforming potential of masks are scant and unsystem- 
atic. This kind of investigation could however nicely fit within a recent strand 
of psychological research on the so-called “enclothed cognition”, whose core 
theoretical bet can be summarized as the attempt to disprove the saying “the 
cowl does not make the monk” (Adam and Galinsky 2012). However, a recent 
replication failure of Adam and Galinsky’s pioneering experiment suggests cau- 
tion in interpreting the earlier results (Burns et al. 2019). 
https://research2reality.com/health-medicine/visual-perception-pathways- 
vision-facial-recognition-covid-pandemic/. 


2 Contagious faces 


Coping digitally with the 
pandemic by means of memes! 


Gabriele Marino 


1. Introduction: the facepalm 


Star Trek: The Next Generation season 3, episode 13, titled “Déjà Q”, 
first aired in the United States on February 5, 1990. The plot is typical: the 
crew of the starship Enterprise has to solve a problem (planet Bre’el IV’s 
moon is about to go out of its orbit), and the solution is complicated by an 
unexpected event as an old enemy named Q (portrayed by John de Lancie) 
suddenly reappears—the déja vu alluded to by the pun in the title. “Déja 
Q” is not particularly memorable, but it shows a moment made memo- 
rable by the strange mechanisms of the Internet. Captain Picard (iconi- 
cally portrayed by Patrick Stewart) sits in his armchair, legs crossed, elbows 
propped on the armrests; reluctantly convinced to help his nemesis (“In all 
the universe, you’re the closest thing I have to a friend, Jean — Luc”, Q tells 
him, surprisingly), he tilts his head slightly forward and, left thumb at his 
temple, covers his eyes, nose, and part of his sullen mouth with the other 
fingers of his left hand. The sequence lasts only a fraction of a second, but 
it has been frozen as a cult. 

“The physical gesture of placing one’s hand across one’s face or lowering 
one’s face into one’s hand or hands, covering or closing one’s eyes” (Wiki- 
pedia)? was obviously not invented by Star Trek (think, for instance, of the 
famous sculpture Cain by Henri Vidal exhibited in the Tuileries in Paris 
since 1896), nor is it exclusive to humans (apes such as the Mandrills have 
it too, say ethologists) but has been baptized more recently: the neologism 
“facepalm” was coined on Usenet (the proto-forum where emoticons were 
created in 1982) around 1996, according to the Oxford English Diction- 
ary, and has been widely used on the Internet since at least 2007 when its 
prototypical visual token, Picard’s, was first uploaded on YouTube as a 
short video fragment.? 

Picard’s facepalm has become an Internet classic: it entered our online 
communication habits by virtue of its expressive synthesis, its ability to 
capture an entire pathemic—that is, passional, emotional—state in one 
essential visual configuration. In this respect, the facepalm does participate 
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in the same process of digital “writing of the face” (faces turning into writ- 
ing characters) or “facialization of writing” (writing becoming face-like 
tokens) initiated by emoticons.* Picard’s facepalm functions both as an 
Internet cliché, a ready-made means of commenting on something (just like 
the cartooned faces of Nicolas Cage, Jackie Chan, and Barney Stinson or 
the Microsoft Paint-designed “rage comics” of 4chan),° and as an icon, a 
figure that we can potentially paste onto anything (like the faces of Chuck 
Norris or Kim Jong-Un). 

The facepalm is interesting to us for at least three reasons: (i) it is an 
Internet meme, (ii) it shows a face but not only that, and (iii) we use it to 
express a certain mood - frustration. 


2. Pandemic frustration 


During the new coronavirus/COVID-19 pandemic we have all been locked 
up, so that our most important object of value—what we aspire to as 
subjects—was literally outside the window, outside the door: we just 
wanted to go out whenever we wanted, as it was before.® All the people 
who lived in such a situation were dismayed and upset because they had 
to do something that was quite difficult to accomplish suddenly: radically 
change their habits and move out of the realm of normality. Trapped inside 
their homes, people were sitting in front of their computers, tablets, and 
smartphones, trying to grasp the unexpected, to understand something they 
had never experienced before, something beyond human comprehension— 
a “hyper object”, as English philosopher Timothy Morton (2013) puts 
it. In confronting this uncanny the only interpretative framework avail- 
able came from the fictional world: books, films, comics, and songs, all 
posted as Cliff Notes-like lists onto social media with a preference for 
the catastrophic, the post-apocalyptical, the dystopian—the epidemical: 
Steven Soderbergh’s thriller Contagion, 2011, and David Quammen’s pop- 
ular science essay Spillover, 2012, became the favorites to many. We tried 
to understand what we went through by putting it in a framework that 
addressed thematized it.” We all experienced “the feeling of being upset 
or annoyed as a result of being unable to change or achieve something” 
(Google Dictionary)’; in a word, we were all frustrated. 

Frustration is a tensive pathemic state that implies a projection toward 
something where the intended action can remain only an attempt: you 
can see and almost touch the desired object of value, and yet you cannot 
grasp it; you lean forward to it, but the narrative program centered upon it 
(i.e., getting the object) is destined to remain unfulfilled.” Normality seems 
to be just outside the window, outside the door, but it is not. Frustration 
has a durative aspectuality; in other words, dissatisfaction (in many ways 
a synonym for frustration) is not episodic but rather long-lasting. We 
have been waiting (e.g., for new dispositions from the government, for 
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a vaccine, for a cure, and so on); we had to “Keep Calm and Carry On” 
(as the famous 1939 prewar British motivational poster reads); at first it 
seemed to be only “a moment” but soon became “a period”: a new “age of 


anxiety”,!! a stasis and silence reminiscent of the Cold War, a war of attri- 


tion. The clock beats a “time out of joint”: every day is a Groundhog Day, 
an endless Sunday of work where any micro-variation becomes an event. 
We have been wedged between a present perfect that seemed to be a simple 
past and that we longed for as if it were a perfect past (“do you remember 
how nice it was to be crammed in the subway?”) and a near future that no 
longer seemed a future perfect and that we did not know when it would 
actually become present (“we will be back to normal”, “everything will 
” 6 


change”, “we will have to get used to medical face masks, like in Japan and 
China”). Frustration has become our semiotic state of mind." 


3. The virality of the virus 


Recent history has already provided communication scholars with events 
that have taken place and been reported, lived, and appropriated on social 
media, to such an extent that the corresponding hashtags have set the 
global agenda for the last decade: #ArabSpring (2010), #Occupy WallStreet 
(2011), #BlackLivesMatter (2013), #JeSuisCharlie and #Love Wins (2015), 
#Brexit and #MakeAmericaGreatAgain (2016), #MeToo (2017), and #Cli- 
mateStrike (2018). None of these events, however, had the truly global 
scale of the pandemic that, as far as we are concerned, began in Wuhan, 
China, in December 2019. This is perhaps the first time, as Slovenian phi- 
losopher and pop icon Slavoj Zizek, among others, pointed out," that the 
whole world feels like it is “in the same boat”, in the same frame, and 
participating in the same narrative that is happening, being shown, and 
told in real time. The pandemic—the use of this not-so-common word has 
increased by more than 57,000% in 2020 (Rajan 2020)—naturally has its 
own hashtags: #StayAtHome and #StaySafe. 

Indeed, the keywords of the virus are more complex, so much so that, for 
the first time, the Oxford English Dictionary did not choose a single “word 
of the year” but produced a report on the many words—including mottos, 
mantras, and other linguistic tokens such as “bushfires, COVID-19, WFH, 
lockdown, circuit-breaker, support bubbles, keyworkers, furlough, Black 
Lives Matter and moonshot” (ibidem)—that marked the “unprecedented” 
year 2020. We were barraged with numbers, data, bulletins, maps, charts 
(with their curves that were never flat enough), and decrees (with their 
idiosyncratic interpretations). Despite all the appeals for an anti-panic, 
anti-alarmistic tone in communications, we had the media screaming the 
news. Conflicting opinions proliferated with “experts” commenting on 
all the numbers, the data, and the news, trying to interpret them in the 
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only possible way: by forcing them, by making them say more than they 
could. Having difficulty reading the present, we projected ourselves into 
the future in a paradoxical accordance with the pragmatic semiotics of 
American philosopher Charles S. Peirce, according to which the meaning 
of something lies in its consequences"; opinions thus became predictions, 
forecasts, and prophecies—a particularly successful literary genre in times 
of crisis. 

We know the limitations of the metaphor of contagion when applied 
outside the biological realm, yet we know that the virus is not merely 
the virus itself but rather its discourse: an object and the words that 
gather around it and define it in the first place.!” The virus is not only an 
RNA nucleus whose sole function is to replicate itself but also a seman- 
tic monad (a revolving core of meaning) that affects meaning-making; 
the virality of the virus is twofold, both biological and cultural. The 
pandemic was overlaid with the so-called infodemic: the hypertrophic, 
spasmodic spread of conflicting information that eventually poisons the 
communication ecology and creates total “semiotic confusion”.'® More 
than any other viral content that can spread online, the virus polarizes; 
it was downplayed or even rejected and denied; it was judged as a turn- 
ing point in human history, the final signal of the capitalistic apocalypse. 
It triggered conspiracy theories (via private groups on instant messaging 
applications such as WhatsApp or Telegram—the next frontier of com- 
munication studies) as well as accurate yet unverifiable etiologies (the 
virus would have been caused by wildlife exploitation, deforestation, 
anthropization, climate change, and so on). To wear the mask, or not to 
wear the mask, that is the question: the virus and the ways to deal with 
it turned out to be a reagent, an axiological (i.e., related to values and 
evaluation criteria) identity swab. Everyone confronts the virus with their 
own semiotic resources, including humor. It is easy to make fun of how 
the virus radically changed the lives of millions; a trivial example: for 
months, Tinder users were deprived of their object of value as the dating 
app was forced to transform into a simple chat. At the same time, it is just 
as easy to see the cases when, paradoxically, nothing changed: doctors in 
hospitals, truck drivers on their vehicles, and freelance graphic designers 
who have always worked from home on their MacBooks have always 
been exactly where they were before the pandemic, though certainly not 
under the same conditions. 


4. Viral memes 


Every discursive wave, and especially the ones so invasive that monopo- 
lize the public sphere, generates Internet memes: pieces of media content— 
mostly catchphrases, captioned pictures, and videos (cut-and-paste, 
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mashups, remixes, and re-enactments)—designed according to a template 
or formula so that they can be appropriated and re-semanticized ad libi- 
tum (their meaning can be adapted according to context and need), usu- 
ally with a playful, funny, humorous, parodic, or satirical connotation. 
We create and share memes to talk about ourselves under the guise of 
talking about current events: when something happens and turns out to be 
big, it becomes memes and every community has its own way.” The pan- 
demic produced not only the infodemic already mentioned but also a true 
“memedemic”; it has never happened before that a single event has been 
the focus of such extensive memetic production on a global scale. We 
have seen similar images pop up everywhere, on every feed of social media, 
from everywhere; a case in point is the images of the war-like scenario of 
empty supermarket shelves with peculiar diatopic variations (i.e., differ- 
ences due to geographical variation) that should be scrutinized by soci- 
ologists and psychologists (while eggs disappeared in Bologna, Italy, toilet 
paper ran out in the United States). And this is exactly what is happening. 
Such a vast semiotic production centered on the virus as a global unify- 
ing isotopy—that is, semantic recurrence (the virus is the common thread 
of all these media tokens)—offers scholars a unique opportunity to study 
communication, culture, and contemporary mythologies in a transcultural 
perspective. 


4.1. Two early researches on “coronamemes” 


Giselinde Kuipers, a sociologist and humor studies specialist based at the 
Catholic University of Leuven, Belgium, and Mark Boukes, who teaches 
communication sciences at the University of Amsterdam, Netherlands, have 
launched a study on “Humor during the global Corona Crisis”, with the 
goal to investigate the social value of what we may call pandemic humor 
within the wider framework of the so-called disaster humor: not all of us 
would readily accept and positively respond to a joke about 9/11 and Twin 
Towers (especially not in the immediate aftermath of the event), but some 
might benefit or might experience a kind of relief by addressing the trag- 
edy in this way. The two scholars have set up a webpage through which 
they collect spontaneous submissions of jokes, images, and memes about 
the coronavirus (“coronamemes”). According to Kuipers, as of April 2020, 
1,357 users from about 50 countries have contributed to this research?'; as 
stated in the only publication to date (as of September 2023) resulting from 
the research (Kuipers and Boukes 2021; a popular science short paper), 
more than 12,000 humor samples from 81 countries had been collected by 
July 2020 with the participation of researchers—including myself—from 
25 countries. The researchers identified five phases of pandemic meme pro- 
duction that can likely be called the “first global humor cycle”. “Although 
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the timing is somewhat different across countries, we see the same phases 
around the world” (ibidem): 


e The Chinese virus (where the pandemic is seen as something external, 
affecting only “the others” and accompanied by racism and xenophobia); 

e Classic disaster humor (where the situation is connoted as movie-like 
and, thus, surreal, and online users exploit the possible cultural associa- 
tions with the fictional world); 

e The virus hits home (where the pandemic becomes an internal affair 
spreading “panic and fear, expressed through consumption”—as is the 
case of the aforementioned eggs and toilet paper—and, at the same time, 
panic and fear are being mocked and ridiculed); 

e The quarantine and lockdown (where people have to deal with physical 
limitations, smart working, routine, boredom, and frustration); 

e What I would call “the normalization of the abnormal” or, as many did, 
“the new normal” (where alienation and isolation make their way, lead- 
ing to a persistent and “general sense of doom”). 


By June 2020, Kuipers and Boukes note, “[O]ther events made the 
headlines—notably the Black Lives Matter protests” (an identity, political, 
and discursive comeback) so that the “attention had dispersed, and the 
period of strong, undivided global focus was over. And with it, the first 
global humor cycle”. 

A group of researchers at the IUSS (School for Advanced Studies in 
Pavia, Italy), at the intersection of cognitive science and linguistics, coordi- 
nated by Luca Bischetti, Paolo Canal, and Valentina Bambini, has launched 
a study on “viral humor” based on a questionnaire, whose participants 
are asked to rate jokes, cartoons, and actual memes with both coronavirus 
and normal themes.” The aim is to investigate the limits of humor in situ- 
ations such as the pandemic and to explore the possible mechanisms that 
make a certain content be considered funny or, on the contrary, disturbing 
(in Internet jargon, we would say, LOL versus cringe, i.e., laughing out 
loud versus “a feeling of embarrassment or awkwardness”). The research 
has so far produced two very similar publications, one in English and one 
in Italian (Bischetti et al. 2021a, 2021b), based on the participation of 
1,751 Italian compilers reached between March 18 and March 30, 2020 
(namely, shortly after the first nationwide lockdown issued by the Italian 
government, which occurred on March 8). According to the researchers, 
“COVID-19 humor is not rated as funnier than humor forms paying on 
heterogenous themes”, as it seems to lack “a ‘signature’ of funniness”, 
while it “displays a mark of aversiveness”; in other words, “not everybody 
enjoys COVID-19 humor” and, as a matter of fact, “with increasing age 


and in women, COVID-19 humor was judged as more aversive”.”° 


28 Gabriele Marino 


The virus spread, and so did humor as it is today: posted online and 
made of memes. On the one hand, as we have seen, a kind of shared global 
feeling emerged, reflected in the use of meme characters that embody 
the same narrative patterns (De Saint Laurent et al. 2021); on the other 
hand, we could simultaneously observe a proliferation of micro-contextual 
humor, that is, humor related to geographically and culturally specific top- 
ics and issues. This trend has been explored in depth in several papers (e.g., 
Sebba-Elran 2021 for the Israeli context; Chtopicki and Brzozowska 2021 
for Poland; Bracciale 2020; Barile and Panarari 2021; Murru and Vicari 
2021 for Italy) and, for example, in edited volume (Mpofu 2021) focusing 
on how social media humor mediated the pandemic in the so-called Global 
South (countries such as Brazil, India, Kenya, Nigeria, South Africa, and 
Zimbabwe).** Social media users all responded to the virus together and in 
similar ways but made the most of the cultural specifics they had at their 
disposal. 


5. Laughing or smiling? 


It may seem paradoxical that such a terrible moment in history generated 
so much playful digital buzz; in fact, this would fit a strategy of coping with 
the virus and its aftermath, such as that early reported in China (Koetse 
and Barnes 2020). Psychological studies would show how “Internet memes 
related to the current COVID-19 pandemic may tentatively serve as cop- 
ing mechanism for individuals experiencing severe symptoms of anxiety” 
(Akram et al. 2021).?° Paralyzed by the lockdowns, we were left with few 
semiotic resources to express ourselves; in this framework, memes are of 
particular importance precisely because of their stupidity; they are perhaps 
necessary precisely because of their ephemerality as they represent the most 
widespread means through which a particular (more or less globalized) 
culture realizes its own homeostasis: the metabolization and negotiation 
of contents and meanings in discourse. To better scrutiny, memes do not 
merely perpetuate the virality but rather put it between “quotation marks”; 
likewise, the so-called coronamemes do not talk about the virus but about 
our obsession with it: they are not the means to semiotically propagate 
the virus further but the parody of its Internet renditions. Whereas viral- 
ity is broadcasted and monolithic, memeticity may be fractal and polyvo- 
cal. Memes, in this perspective, can be understood as what French Jesuit 
anthropologist Michel de Certeau (1980) would call digital “practices 
of everyday resistance” and French philosopher Michel Foucault (1984) 
would call digital “technologies of the self”. While history always repeats 
itself twice, according to the famous Marxist adage, we can front tragedy 
and turn it into farce through silly tokens like memes: we can create meta- 
structural descriptions (grammars, according to Russian semiotician Yuri 
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Lotman)* in which we are not the passive extras but the active directors of 
the bad movie we are living in. 

In one way or another, we have asked ourselves: can we already laugh 
at COVID-19? Is it still much too soon? Some believe that we must laugh 
at the virus as soon as possible precisely because it would not be the case: 
in this way we can to some extent reduce its scale; we can try to normalize 
such an overwhelming phenomenon. To avoid our global semiotic silence, 
we can try—some suggest—to laugh about the virus and fill any possi- 
ble meme template. Controversial musician and media personality Kanye 
West, proverbial for his erratic laughter as well as for the complete lack 
of facial expression at public events, appears in a five-panel meme from 
March 2020 that rearranges some old pictures of him for the occasion: 
under the headline “Laughing at corona memes like” we see five differ- 
ent pictures of West; he is always smiling, even though his facial expres- 
sion is less and less amused over time, from “February” to “March 1” to 
“last week” to “this morning” and, finally, to “a little later this morn- 
ing”. Thanks to the semiotic act of artificially altering reality, pictures, 
bodies, and faces, we may find a temporary, vicarious digital way out: 
we can laugh less, maybe just smile, to cover up a situation that is get- 
ting worse. German philosopher Helmut Plessner (1950), founder of philo- 
sophical anthropology along with Max Scheler and Arnold Gehlen, has 
written extensively on the difference between “laughing” (Germ. Lachen) 
and “smiling” (Lächeln), two expressions that can easily be confused (or 
reduced one to the other). While “laughing” would manifest the uniquely 
human “eccentric positionality” in the respect of being an extreme limit — 
case of behavior, expressing the brokenness of human beings’ relation to 
their body (“crying” would function in the same way), “smiling” would 
embody such “eccentricity” in a much more subtle way: it is an ambiva- 
lent mark of awareness that can be either positive or negative, as it is not 
strictly tied to a particular emotion. In other words, for Plessner, every 
smile is always the ambiguous smile of Gioconda. 


6. Face and face mask 


As we have seen, the pandemic reconfigured our habits, our proxemics 
(i.e., our use of space), and our kinesics (our body movements in space). 
It also “disfigured” us: we could not go to barber shops or hairdressers 
(since they were closed), and yet we were told to keep hair short and cut 
mustaches and beards (since they were potential vehicles of contagion). 
Above all, the pandemic forced us to always wear the medical face mask, 
which became a true collective and global obsession, both positive and 
negative. The mask was appropriated in the most diverse ways, with peo- 
ple “wearing rather peculiar items on their faces” and parodic imitations of 
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them “done for fun by individuals stranded at home” or “dressed in most 
peculiar, attention-grabbing attire” (Dynel 2020: 184-185). The term 
“mask shaming” spread from early 2020 to denote social deprecation for 
wearing or for not wearing the medical face mask in public spaces—a case 
of polysemy, which linguists define as enantiosemy, where the same word 
means its own antonym (a word with the opposite meaning) depending on 
context (Corbolante 2020).”? 

The mask is a thin layer of fabric and yet a powerful device, physical 
and semiotic at the same time; it would prevent contagion as much as it 
would invalidate face-to-face communication as we know it by getting in 
the way of our facial expressions. It would dismantle the idea of the face 
as “the epiphany of the Other” proposed by Lithuanian-French philoso- 
pher Emmanuel Lévinas (1961) and revisited by contemporary thinkers 
such as Italian Giorgio Agamben (2020), who not coincidentally happens 
to be a fierce opponent of masks and social distancing.*° The mask testi- 
fies the “faciality” (Fr. visagéité) theorized by French psychoanalyst Félix 
Guattari (1979)?! as a means of imposing control over our bodily social 
life—something that Foucault (1976) would call “biopolitics”. This small 
piece of fabric that people in the West were forced to wear in order to cover 
their mouth and nose, for the very first time, revealed the existence of at 
least two opposing ideologies of the face: one ontological and essentialist 
(and conservative; the face is being obtruded and disrupted by the mask, 
and, without their full face, human beings are no longer fully human) and 
the other semiotic and constructivist (and transhumanist, since the mask is 
just an appendage of the biological face and human nature is defined by the 
need to go beyond the mere biological level).* 


6.1. Pandemic meme faces 


After the virus, new memes were created. And the old ones were revamped 
to fit the Zeitgeist; Google Trends shows an almost perfect homomorphism 
and synchronicity between the global popularity peak of search terms like 
“coronavirus” and “facepalm” around mid-March 2020. Correlation 
does not imply causation, says the old adage; nevertheless, the facepalm 
“in the time of virus” shows us how—to put it with Italian satirist Ennio 
Flaiano—“the situation is grave but never serious” so that its cartoonish 
and yet realistic digital mimicry has been capable to thematize the surreal, 
dramatic scenario. Due to the widespread World Health Organization pre- 
scription not to touch one’s eyes, nose, or mouth to prevent infection, the 
facepalm has been forced to give up hands to convey its message. So its 
message changed. 

Many meme formats rely on comparisons, and this is also the case with a 
picture posted in April 2020 on Imgflip, a website for creating and sharing 
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images (specialized in memes and animated GIFs) launched in 2008, where 
the traditional facepalm is updated “in the time of virus”.** Picard’s face- 
palm is crossed by a red “No entry” sign and introduced by the follow- 
ing text: “ATTENTION IMGLIP COMMUNITY: UNTIL FURTHER 
NOTICE, USE OF PICARD FACEPALM TEMPLATE IS SUSPENDED”. 
The new pandemic-compliant facepalm is directly below and palm-free; it 
is identified by a green check mark (“Yes, this is ok”) and is introduced by 
the text “ALTERNATE NO FACE TOUCHING TEMPLATE”, with the 
final prompt “PLEASE USE THE ABOVE IMAGE AS AN ALTERNATE 
UNTIL THIS CRISIS PASSES. THANK YOU FOR YOU COOPERA- 
TION”. The meme is a meta-meme (a meme about memes), and it is play- 
ful, but it is nevertheless evidence of a change in collective behavior IRL 
(in real life) through a change in collective digital behavior. Post-pandemic 
Picard just frowns without touching his face in the slightest. 

Another meme that surfaced on Twitter in March 2020 depicts Sla- 
voj Zizek, who is known to suffer from tics that cause him to constantly 
touch his face when he speaks, being appalled by the anti-contagion pre- 
scriptions; again, the meme is based upon comparison: Zizek reacts to 
the “*Coronavirus outbreak*” (and the resulting invitation “*in order 
to prevent the infection do not touch your nose, eyes or mouth with your 
hands*”) with a typical exaggerated facial expression of his.” Another 
Imgflip meme shows a generic little guy in front of their computer engag- 
ing in a desperate double facepalm; the top caption reads, “SERIOUSLY 
STOP MAKING CORONAVIRUS MEMES” and the bottom one, “IT’S 
NOT FUNNY!”.*° We have created way too many COVID-19 memes to 
make all of them hilarious; we may be still smiling but in the same way of 
good ol’ Kanye. 

The facepalm, which in Western culture is considered the figure of 
frustration par excellence,” shows us that not only the face but the 
whole body is involved in such a pathemic state. The face is a powerful 
catalyst of meaning, but it owes this capacity to its synecdochic (the part 
for the whole) and metonymic (the concrete for the abstract) nature: it 
stands for the body and, thus, for the persona. Semiotics knows that 
meaning is always relational and that any subject of research is relative 
and mobile, just like the perspectives that the researcher must adopt to 
do their job—without condoning nor condemning them. In other words, 
to study the face as a sociocultural entity is not to study the face as a part 
of the body or only from a visual perspective. The face is just the tip of 
the iceberg, the screen onto which we project the form of life to which 
we adhere, and the facepalm, with the hand covering the face only to 
uncover its haptic dimension (i.e., the dimension related to the sense of 
touch), reveals the semiotic mask of frustration—a passion that deeply 
affects our form of life.°® 
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7. Hands 


The pandemic did not mutilate us, but it disabled our hands as if they were 
wounded. We could not touch other people. We were instructed not to 
touch our own noses, eyes, and mouths. And to wash, sanitize, cover, and 
hide our hands (with disposable gloves). We have been told this over and 
over to the point where repetition may have caused quite a loss of meaning; 
this surreal meaninglessness has been attested by memes that synthesize 
the new rituality by, for instance, associating it with OCD - like behaviors 
previously practiced only by literature’s most famous killers: the classic 
Shakespearean scene in which Lady Macbeth frantically washes her hands 
after having committed murder became a surreal, memetic guide to hand- 
washing (see Smith 2020). 

By the time of the pandemic, TikTok, launched in 2018, had already 
become a pivotal social and viral platform, which did nothing but “extend- 
ing the Internet meme” logic (Zulli and Zulli 2020); in early January 2020, 
fears of the outbreak of a Third World War due to the escalating tension 
between the United States (with Donald Trump) and North Korea (with 
Kim Jong-Un) were greeted by themed videos with hashtags such as #WW3. 
After that, TikTok became the social media reference par excellence for 
the pandemic; Khaby Lame, the most followed creator on the platform 
(since June 2022), started his own career, based on a signature hand ges- 
ture mocking the so-called life hacks videos that spread virally due to their 
entertaining complicatedness, during the first Italian lockdown. Among the 
TikTok trending fads we find the infamous handwashing routine: 


Vietnamese officials created a TikTok dance demonstrating the proper 
way to wash hands and engage in social distancing. . . . The dance 
launched a “challenge” where users imitated the hand-washing proce- 
dure, presumably for increased visibility, but incidentally promoting/ 
spreading safe behaviors. 

(ivi, 13) 


In Michelangelo’s frescoes of the Sistine Chapel in Rome, God extends his 
hand toward Adam to give him life; “in the time of virus”, Instagramer 
Shusaku Takaoka photoshopped the scene so that God, wearing dispos- 
able gloves, would sanitize Adam’s hand.” Without our hands we have 
trouble doing, making, interacting—being ourselves. We shake hands and 
hold hands, identifying two distinct regimes of relationality: superficial 
acquaintance and intimate relationship. The hand defines us as active sub- 
jects: metaphors and language in general suggest this by means of etymol- 
ogy (from the Latin manus to “manufacture”, from the English “hand” to 
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“handle”, and so on), outlining the semantics of a metonymy of our own 
agency—we can do things because we have hands. In the incipit of French 
art historian Henri Focillon’s In praise of hands (1934) this “poetry of 
action” is described in vivid terms; hands are underappreciated longtime 
friends that play a leading role far from the vicariousness to which they 
have been historically reduced within a Cartesian framework that juxta- 
poses mind and body. A leading role that in Focillon’s words transforms 
them, not surprisingly, into faces: 


I undertake this essay in praise of hands as if in fulfillment of a duty 
to a friend. Even as I begin to write, I see my own hands calling out 
to my mind and inciting it. Here, facing me, are these tireless compan- 
ions who for so many years have served me well, one holding the paper 
steady, the other peopling the white page with hurried, dark, active little 
marks. Through his hands man establishes contact with the austerity of 
thought. They quarry its rough mass. Upon it they impose form, outline 
and, in the very act of writing, style. Hands are almost living beings. 
Only servants? Possibly. Servants, then, endowed with a vigorous free 
spirit, with a physiognomy. Eyeless and voiceless faces which nonethe- 
less see and speak. 


We know that the opposable thumb is one of the traits that make us bio- 
logically and phylogenetically unique. Some scientific theories place the ori- 
gin of language in the gestures of the hands, meant as the “original signs” 
(Armstrong 1999). Among the earliest examples of visual expression are 
the negatives—the stencils—of the hands in the caves: indexical (marks 
of physical intervention), iconic (they represent the object they stand for), 
and symbolic (concrete figures standing for abstract qualities) signs at the 
same time, thematizing their own manufacture. There are systematic stud- 
ies of the hands when their use is codified, explicitly linguistic: think of 
sign language (and its idiosyncratic variations, such as those collected in 
Munari 1963). There are protosemiotic fields of study that deal with the 
signs of the hand (chiromancy) and parasemiotic fields of study devoted to 
the signs left by the hand (calligraphy and graphology). Hands are the sub- 
ject of iconological studies. In general, though, precisely because of their 
importance, precisely because of their prominence, precisely because we 
cannot do without them, we tend to forget about hands—we take them for 
granted. We think of our hands when we are forced to rethink them: when 
they become a problem, when we cannot use them, when we have to pay 
attention to them, when they are denied to us, and when they cease to be 
“natural”. Just like when they get hurt or wounded. Just like during the 
pandemic. 
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8. Hands and faces 


During the pandemic, pictures of exhausted doctors, nurses, and other 
medical professionals after hours of work went viral, showing the signs 
of medical face masks on their faces and how their hands were abraded 
from continuous handwashing and sanitizing (see Leone 2020d). In order 
to protect them we have to hurt both the face and the hands: they are 
being damaged by a virus that primarily affects the respiratory system—the 
umpteenth proof of the systemic nature of any phenomenon, biological 
and semiotic at the same time. Hands and face seem to be only the different 
parts of one and the same circuit. 

Famous are the passages of French semiotician Roland Barthes (1980) 
about the face immortalized by the photographic act. Some romantic “frag- 
ments” of his (1977) describing hands and fingers are equally memorable. 
Less well known are perhaps the reflections in his Variations on Writing— 
an essay commissioned by the Academic Institute in Rome in 1971, written 
in 1973 and published posthumously only in 1994—in which Barthes talks 
about the hands and the face. 


When the graphic sign first appeared, a new equilibrium emerged 
between the hand and the face (they were liberated simultaneously, one 
with the help of the other): the face acquired its own language (that 
of hearing and speaking) and the hand also acquired its own language 
(that of grasping vision through gestural tracing) [p. 20]. . . . Finally, 
after freeing themselves thanks to each other, from one side, we got the 
hand (the gesture) and its manufacturing functions and, from the other 
side, the face (the word) and its functions of phonation. And what about 
writing? It is undoubtedly a return to the hand. . . . Language returns 
to that part of the body whose liberation had made possible its birth: a 
great dialectical process comes to its closure. Writing is always on the 
side of the gesture, never on the side of the face: it is tactile, not oral. 
[p. 63]*° 


The particular sensibility of Barthes, who here proposes something that 
has been called the “physics of writing” (Lo Feudo 2017), helps us draw a 
micro-map of the complex relationship between face and hands, according 
to a complementarity that is functional and substantial: language becomes 
words through the face; language becomes gestures through the hands in 
writing. 

Italian semiotician Massimo Leone (1999) devotes the last part of his 
analysis of representations of religious ecstasy to hands and, thus, to the 
sense of touch: an “uncomfortable subject”, something difficult to explore 
without first having accumulated an adequate “memory of images”, yet 
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something that also represents “the culmination of many paths of ecstasy” 
(p. 252), especially in the figuration of the “hand on the chest”. As the site 
of contact, of physical contiguity between subject and object, the hands are 
the site of relationality: by playing the role of the enunciational informer- 
actant par excellence (the one who indicates, who makes it visible, who 
draws attention to something),*' the hands increase their “semiotic ver- 
satility” in “conjunction with other equally flexible elements, such as the 
face” (p. 257), becoming a powerful “instrument of ostension” (p. 255). In 
other words: when these deictic “figures of the body” point to something, 
they would like to touch it, and even when they actually touch something, 
the hands are pointing to it. Leone (2021b) returns to this ongoing dialec- 
tic between hands (and especially fingers) and faces to address the issue 
of recognition and identity in the age of “digital AI forensics”: “modern 
forensics realizes that the face can mislead and, inspired by eastern models 
(China, Japan, India), adopts fingerprinting. In the digital era, however, 
fingerprinting first goes digital, then it is increasingly replaced by facial 
recognition” (ivi, 579). 


9. Conclusions: parafaciality and frustration to the square 


Hands can “speak” on their own. Quite often they speak along with the 
face, amplifying or silencing it, by making it explicit, reinforcing its commu- 
nicative attitude. Without them, the face still signifies, but without them it 
often simply communicates weaklier. The face can be enriched and detailed 
by a variety of complementary elements—we can talk of “parafaciality” 
(paratextual faciality, from Ancient Greek preposition zapda, “beside, in 
adjacency”). Some elements can be defined as peritextual regarding the 
face since they are characterized by spatial continuity with the face; think 
of cosmetics, jewelry, tattoos, headpieces, and, of course, masks—we can 
speak of “perifaciality” (peritextual faciality, from zepi, “around, near, 
about”). The face can also be epitextualized (from ézi, “in addition”), that 
is, provided with complements that intervene on it but do not have direct 
continuity with it**—we can speak of “epifaciality” (epitextual faciality). 
This is the case with the hands. They are autonomous, they are far from 
the face, and yet we cannot help but put them onto it: through them we can 
manipulate the face; we can make this physical part of the body a powerful 
means to express and show our emotions, our complex pathemic states. 
Thanks to the hands we can appropriate the face and turn this biological 
surface into our semiotic mask.** With our hands we shut our mouth, plug 
our ears and nose, and shield our eyes or the entire face; we do so even 
when the meaning of our expressions is perfectly readable in itself.** Still, it 
is not enough. We may be “desperate”, and this may already be conveyed 
through our grimace, but we are telling those who look at us that we are 
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desperate only when we “tear our hair out”, whether it is intentional or 
not, whether we have hair or not—as in the case of the bald Captain Picard. 

A small piece of digital media content like an old Internet meme man- 
ages to make us aware that we are forced to radically rethink our idea of 
how our persona coheres in the public sphere: the facepalm proposes to us 
a surprising redefinition of what our face was and can or cannot be. The 
amplifying epifaciality of the hands is perfectly visualized in action, albeit 
in absentia, within the facepalm meme modified “in the time of virus”: 
the prototype, censored from any possible use of the hand, is profoundly 
weakened. Picard is now frustrated to the square: frustrated by its own 
frustration. Without his hand, the Captain looks frustrated by the fact that 
he cannot be frustrated enough. Indeed, during the pandemic, we are not 
even allowed to be as frustrated as we would like to be. Yet we can choose 
to keep our semiotic élan alive: we can hijack language and divert it from 
our hands back to our face. And laugh about it. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 

2 “Facepalm”, Wikipedia, https://en.wikipedia.org/wiki/Facepalm (entry created 
in 2008). 

3 “Mortal Q”, YouTube, May 22, 2007, https://youtu.be/x012BnK Wi3g?t=57 
(at 00:57 seconds c.ca). 

4 For a semiotic approach to emoticons and emojis, see (Marino 2022a). 

5 4chan (www.4chan.org/) is the best-known image board in Western Internet 
culture, founded in 2003 by then 15-year-old American nerd Christopher 
Pool aka moot; it is generally considered the “factory of Internet memes” par 
excellence. 

6 For the semiotic notion of “object of value”, see Greimas and Courtés (1979, 
ad vocem, 216-217). Note: in-text and footnote page numbers refer to the 
translation when included in the list of references. 

7 For the semiotic notion of “thematization”, see Greimas and Courtés (1979, ad 
vocem, 344). 

8 As of September 2023, the definition in Google’s Search Engine Results Page 
(SERP) is provided by Oxford Languages; http://bit.ly/30Z7GD3. 

9 For “tensive semiotics”, see Fontanille and Zilberberg (1998). For the semi- 
otic notion of “narrative program”, see Greimas and Courtés (1979, ad vocem 
[“program, narrative” ], 245-246). 

10 For the semiotic notion of “aspectualization”, see Greimas and Courtés (1979, 
ad vocem, 18-19). 

11 The proverbial title of a 1947 poem by British-American modernist poet W. H. 
Auden. 

12 Title of American science fiction writer Philip K. Dick’s 1959 novel (in which 
the fifties are recounted as if they were a distant time, idealized and stereotyped 
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while deformed through the lens of SF narrative), inspired by a verse from Wil- 
liam Shakespeare’ Hamlet (1600 c.ca). 

The classic reference for the “semiotics of passions” is Greimas and Fontanille 
(1991). See also Greimas’ seminal work on the lexical semantics of nostalgia 
(1986). 

In an interview with RT News, uploaded on YouTube on March 15, 2020 
(https://youtu.be/HabyJi66l0w, search via WebArchive). 

A case in point is Italy, where the vagueness and ambiguity of government de- 
crees have become proverbial (TheLocal.it 2020). 

The famous passage by Peirce on the relation between meaning, action, and 
future is CP 2.86 (from 1902). 

For the semiotic notion of “discourse”, see Greimas (1966). 

The term is not used here in the same sense as in language contact studies (e.g., 
Caunce 2013). 

For a semiotic approach to viral contents and Internet memes, see Marino 
(2015, 2022b). The best introduction to the topic of “memes in digital culture” 
is still by Shifman (2013), who was involved in the research of Kuipers and 
Boukes. 

It is impossible to provide even a selection or glimpse of such material here; 
as a rough guide, take a look at the Know Your Meme entry (the wiki en- 
cyclopedia of viral phenomena and Internet memes, launched in 2007) dedi- 
cated to the COVID-19 pandemic: https://knowyourmeme.com/memes/events/ 
covid-19-pandemic. 

Personal communication, via email, March 23, 2020. The URL of the sub- 
mission webpage (still active as of January 2023) is https://uvacommscience. 
eu.qualtrics.com/jfe/form/SV_eOGzCgEHtNWSPZX. 

The URL to access the questionnaire (https://scuolaiuss2019.limequery. 
com/635545) was active only during the data-collecting period (i.e., March 18- 
30, 2020); see Pattarini (2020) for a description of the project. 

Researchers suggest that “individuals using humor to cope with uneasy circum- 
stances judged COVID-19 humor as funnier and less aversive”, whereas one 
could reverse the perspective and claim that individuals who found COVID-19 
humor funny do use humor to cope with uneasy circumstances. 

For a synthetic review of “humor in the age of coronavirus” from an Anglo- 
phone perspective, see Olah and Hempelmann (2021). 

The study is based upon the response of 160 individuals to coronavirus-related 
memes, 80 with clinical anxiety, and 80 as a control group (i.e., without clinical 
anxiety). 

Lotman (1971). 

“Laughing at corona memes like”, a Kanye West pandemic meme circulated in 
March 2020, an instance of which is displayed at “Covid-19 Archive. A Jour- 
nal of the Plague Year” (https://covid-19archive.org/s/archive/item/9716), a 
non-profit website aimed to “document, curate, and preserve experiences of 
the Covid-19 pandemic for the historical record”. Original West photographs 
date back to at least 2015. 

Italian semiotician Bruno Surace’s chapter in this same collection addresses dis- 
figurement from a semiotic perspective, focusing on horror films and digital 
urban legends (such as the so-called Creepypasta stories and the infamous char- 
acter called Slender Man); see also Surace (2021a). 

See also the entry “Mask Shamed” in Urban Dictionary, October 1, 2020, 
www.urbandictionary.com/define.php?term=Mask+Shamed. 
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Agamben’s philosophy of the face is also deeply indebted to that of Martin 
Heidegger. 

Guattari later developed this concept further in his writings with French phi- 
losopher Gilles Deleuze. 

To define themselves, human beings have always redefined themselves. The ear- 
liest depiction of humans trying to go beyond their biological nature is perhaps 
to be found in the Bible, Genesis 3, where Adam and Eve finally reveal their 
sinful nature. God allows a kind of “improvement” (he gives them clothes) 
but still sets an insurmountable limit for them: death because they were never 
meant to be immortal. 

GoogleTrendsresultsfor “coronavirus” and “facepalm” between November 2019 
and June 2020: https://trends.google.it/trends/explore?date=2019-11-01%20 
2020-06-30&q=coronavirus, https://trends.google.it/trends/explore?date=2019- 
11-01%202020-06-308q=facepalm. 

The facepalm “in the time of virus”, a meta-meme uploaded on “Imgflip” (https:// 
imgflip.com/i/3v2pvx) in April 2020 that has been viewed 71,000 times as of 
September 2023. 

https://twitter.com/dubravka/status/1240027570663546880. 
https://imgflip.com/i/3sobtl. 
https://en.wikipedia.org/wiki/Facepalm#Popular_culture. 

For the semiotic notion of “form of life” (a philosophical term introduced by 
Austrian philosopher Ludwig Wittgenstein), see Fontanille (2015). 
@shusaku1977, Instagram, March 16, 2020, www.instagram.com/p/B9yd- 
Wnpel9/. 

Page numbers refer to the Italian edition; the translation is mine (see list of 
references). 

For the semiotic notion of “actant”, see Greimas and Courtés (1979, ad vocem, 5-6). 
Fontanille’s theory was thoroughly presented in his 2004 book in French; for an 
English summary, see Fontanille (2013). 

The terms “paratext”, “peritext”, and “epitext” are drawn from the theory of 
transtextuality proposed by French narratologist Gérard Genette (1982). 
What we simply call “the face” is in fact a threefold semiotic apparatus: we 
have the biological part of the body (Lat. facies, Eng. face), which may or may 
not show our emotions through physiognomy (vultus, facial expression), just 
as it may or may not be clearly visible to the public (visus, visage). For such a 
semiotic framing of the face, see Magli (1995) and Marino (2021). 

See the classic triptych of emojis showing the so-called three wise monkeys, 
respectively, meaning “see no evil, hear no evil, speak no evil”, as a call to 
discretion, willful ignorance, or self-censorship. In Munari’s visual dictionary 
(op. cit.) we do find facial expressions reinforced by hand gestures but not the 
facepalm. 


3 Uncertain faces 


An investigation into visual 
forms for communicating 
otherness! 


Cristina Voto 


1. Introduction: How to do things with facial images 


The performativity of the artificial face when communicating the meaning 
of otherness is at the core of these pages. Before starting, I consider it appro- 
priate to establish a few premises. In what follows, speaking of performa- 
tivity implies considering those social exchanges that transmit knowledge 
and memory, as in the case of the sense of identity and otherness, through 
repeated actions or “twice behaved-behavior” (Schechner 2002: 29). In a 
dialogue with the proposals that performance studies (Diana Taylor 2016; 
Schechner 2002) have introduced into humanities, performativity is here 
understood as an epistemic lens, an efficient framework for the analysis of 
effects of meaning such as the process of identification, the fixing of iden- 
tity normativity, or the exercising of practices of resistance. Understand- 
ing these processes as performances suggests that the performative nature 
of things can also function as an epistemology: as an embodied practice, 
the identification of performativity always offers a certain form of situated 
knowledge. 

Stuart Hall’s reflections (1996) on cultural identity as a dynamic process 
resonate with this perspective: he refutes existentialist conceptions of iden- 
tity while making strategic and positional, thus performative, use of it. Based 
on this background, I also refer to identity as the result, whether by accept- 
ance or rejection, of the load of meanings that collective writings deposit on 
the skin of each of us. Identity, then, is not to be thought of as a fixed ontol- 
ogy but as a performative space of tension that results from the intersection 
of different aspects — such as those related to ethnicity, class, gender, and so 
on — through which to understand the process of identification. Retrieving 
the lesson of intersectional feminism (Crenshaw 2017), thinking in terms of 
intersectionality allows us, in fact, to recognize the performative nature of 
identity, turning this category into a useful tool for mapping out how each 
identity experience is the result of situated effects of oppression and/or privi- 
lege. Within this perspective, thinking of the process of identity construction 
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means not so much referring to individuation according to subjectivity but 
to a dynamic of identification by means of which the subject relates to oth- 
ers, producing a sense of belonging to a community. 

During identity performances, the Other — the alter - emerges and so 
does the ideological apparatus that shapes otherness. In our culture, one of 
the greatest areas of conflict in this confrontation has been the reproduc- 
tion of facial images as visual artifacts where identity, and thus otherness, 
can be portrayed and communicated. For this reason, in what follows, 
I consider the visual reproduction of the face as a commutative project 
capable of conveying meaning effects concerning the identification process 
and that of othering. Within this viewpoint, it is possible to affirm that even 
though our every face is unique, it is precisely on the frontal surface of the 
head that facial trends can be established, also based on the sociocultural 
expectations which shape aesthetic models in accordance with parameters 
of belonging to a community (Leone 2021c). Maybe this is the reason why, 
as a visual artifact, the face has always been meaningful throughout his- 
tory: “making one’s or others’ face(s) present in a distant space or in a 
distant time through visual simulacra is an old habit of the species” (Leone 
2020a Digital Cosmetics: 551). Nevertheless, the introduction of photog- 
raphy and the possibility of the mechanical visual reproduction of the face 
significantly changed the meanings and uses of the surface par excellence 
of identification in both an honorific and a repressive way. As affirmed by 
Allan Sekula in his fundamental essay “The body and the archive”: 


On the one hand, the photographic portrait extends, accelerates, popu- 
larizes, and degrades a traditional function . . . that of providing for the 
ceremonial presentation of the bourgeois se/f. At the same time, photo- 
graphic portraiture began to perform a role no painted portrait could 
have performed in the same thorough and rigorous fashion. . . . [P] 
hotography came to establish and delimit the terrain of the other, to 
define both the generalized look — the typology — and the contingent 
instance of deviance. 

(Sekula 1986: 6-7, italic in the original) 


Nowadays, two hundred years after the invention of photography, our 
“iconosphere” is more than ever replenished with visual facial artifacts that 
translate ideological discourses into typification and deviance. At the same 
time, the mechanical nature of images has been updated with automated 
forms of re-producing the face. These automated facial images derive from 
the “automatization of the labor of looking” (Lee-Morrison 2019: 18), a 
labor that outstrips the human faculties, and the massive amounts of data 
that digital society produces. 
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From a historiographical point of view, photography before all other 
images assured adherence to the referent by means of the mechanical dimen- 
sion of the device, its distinguishing feature. The referentiality promised by 
the photographic image created a unique link, an indexical one — that is, a 
spatiotemporal connection — between mechanically generated images and 
the realm of veracity. However, our present confronts us with a media ecol- 
ogy capable of challenging the epistemological paradigm of visuality using 
images that look like photography or, should we say, that activate meaning 
effects which inscribe them in the domain of the photographic. In an icono- 
sphere of computer-generated images, artificial vision, and virtual reality, 
photographic images are increasingly in contact with and contaminated by 
digital practices. This change in the mediascape entails a new focus on the 
discursive aspect of the image: asserting that an image is a referent of some- 
thing has become a rhetorical mechanism based on context rather than a 
feature guaranteed by the technical genesis of the image. So how are we to 
approach this visual horizon? 

Some interpreters have already acknowledged the performativity of the 
face as an interface of communication and visual support in the socio- 
cultural processes of identification. On the one hand, Ervin Goffman has 
investigated the ritual character of face-to-face interaction: from his per- 
spective, as rituals are constitutive parts of everyday life, it can be said that 
our daily social fabric is made up of ritualizations that order our facial 
acts. In this sense, interactive rituals appear as embodied on faces whose 
expression is the mastery of gesture, the manifestation of emotions, and the 
ability to present convincing performances before society. For Goffman, 
people show their positions on the scale of prestige and power through a 
social face, an expressive mask that has been lent and attributed by society: 
those people interested in maintaining this social face must take care that a 
certain expressive order is preserved (Goffman 1956). On the other hand, 
it is possible to peruse a whole strand of studies attentive to recognizing 
those sociocultural writings that make the face a fertile medium for the 
recognition of otherness. Examples of this perspective are Joan Riviere’s 
proposal to consider womanliness a mask to be worn (1929), Roland 
Barthes’ interpretation of Greta Garbo’s face-as-object (1957), and Laura 
Mulvey’s feminist reflections on the facial close-up in classic Hollywood 
cinema (1975). In all these proposals, the facial image performs a sense of 
otherness according to gender bias through the media reification of what 
Simone de Beauvoir (1949) defined as the second sex. With these studies as 
background, in the next sections, I turn my attention to the sociocultural 
effects of facial images when produced through a nonhuman agency, tak- 
ing into consideration the capacity of artificial intelligence systems to com- 
municate otherness. 
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2. The semiotics of a precarious form of communicating 
otherness 


After the terrorist attack at the World Trade Centre in New York in Sep- 
tember 2001, the war in Afghanistan, and the circulation of automated 
devices for the detection and recognition of the enemy, the philosopher 
Judith Butler responds to the impacted media environment with a series 
of reflections concerning the conditions of representations of otherness. In 
Precarious Life: The Power of Mourning and Violence (2004) they revise 
the theoretical proposals of Emmanuel Lévinas through a perspective aimed 
at problematizing an ethic of the precarious condition of human life rep- 
resented in their contemporary mediascape. In particular, they dedicate an 
entire chapter to the representations of faces during the war in Afghanistan 
since: “all of these images seem to suspend the precariousness of life. . . . 
They are the spoils of war, or they are the targets of war” (2004: 143, 
italics in the original). In Butler’s philosophical production (1990, 1993, 
1997), the body has already been analyzed as precarious matter negoti- 
ated in space and time by a constitutive vulnerability. Public life, physical 
proximity, and exposure to the gaze of others always modulate the body. 
From this standpoint, and in the warfare that inaugurates the Third Mil- 
lennium, Butler identifies in the openness that marks the body a porosity 
shaped by precarity. The philosopher introduces the latter term to indicate 
those conditions of protection and threat that are maximized or minimized 
by the differential distribution of exposure to damage. Recognizing the pre- 
cariousness of the human ontological condition prompts Butler to investi- 
gate the ethical obligations arising from it in dialogue with the arguments 
presented in Totalité et Infini: essai sur l’extériorité [“Totality and Infinity: 
An Essay on Exteriority”, 1961) by Emmanuel Lévinas. 

In the essay, the Lithuanian philosopher proposes that to conceive sub- 
jectivity, it is necessary to abandon the category of totality and introduce 
that of infinity as a category to encompass the self and otherness in the 
same space-time. Infinity opens the possibility of a subjectivity that in itself 
contains the impossibility of fully embracing it and that comes from abso- 
lute exteriority. From this viewpoint, Lévinas indicates in the human face 
the communicative form of the infinity of otherness. The face is, under his 
perspective, never fully graspable and always ineffable because of its tran- 
sitivity: it allows us to recognize the infinity of humanity as regards those 
not only in whom it has been recognized but also in whoever is performing 
the recognition. With the experience of the Jewish Holocaust at the core of 
his reflections, Lévinas considers the impossibility of looking at someone’s 
face and not recognizing a human being since it is the face that communi- 
cates the infinity of what is human. Embracing this perspective, Butler also 
recognizes in the representations of the faces of the war in Afghanistan 
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diffused throughout their mediascape those of enemies, of heroes, and of 
victims too: “dominant forms of representation [that] can and must be dis- 
rupted for something about the precariousness of life to be apprehended” 
(Butler 2004: XVIII, italics mine). 

In the preface to their Precarious Life, the feminist philosopher writes 
that Lévinas makes use of the face “as a figure that communicates both the 
precariousness of life and the interdiction on violence” (ibidem). From this 
viewpoint, they reinforce the conceptualization of the primacy of other- 
ness sustained by the formulation of a face-oriented structure in an ethical 
relationship. The face offers, in Butler’s words, the infinite unspeakable 
message of precariousness, a porosity effect of constant sociocultural vul- 
nerability and the limitation of linguistic articulations: “the face of the 
Other, and so the ethical demand made by the Other, is that vocalization of 
agony that is not yet language or no longer language” (ibidem: 139). 

Butler places ethics within the public sphere, which implies understand- 
ing ethics not as pre-political but rather as an imperative interest for the 
collectivity (Loizidou 2007). In these pages, I am interested in the same 
ethical concern, which has at its core the capacity of automated facial 
images to socioculturally communicate otherness. This is why I suggest a 
cross-reading of Butler’s and Lévinas’ theses to detect, through a semiotic 
framework, which forms of communication of otherness artificial intelli- 
gence systems may enable. My proposal, thus, is to discern the performa- 
tivity of the face as a communicative form of otherness based on Charles S. 
Pierce’s theories. I suggest recognizing three facial effects that make other- 
ness socioculturally communicable: 


e a surface effect that concerns the qualities of otherness (like a Firstness 
in Peircean theory, CP.1.418)’; 

e an interface effect which interests the material actualization of otherness 
(like a Secondness CP. 1.419); and 

e a meta-face effect that corresponds to sociocultural laws and values of 
otherness (like a Thirdness CP. 1.420).4 


The basting of this schema for the recognition of facial effects finds its 
basis in the validation of a pragmatic perspective. Continuing within Pei- 
rce (CP. 5.400), it is about considering what effects the communication of 
otherness through the face could conceivably have, and these effects, which 
Peirce defines as habits’ (CP 5.538), are the conceptions of the object under 
analysis. In other words, the facial effects have practical consequences on 
the designing of the forms of communication of otherness. According to 
Peirce, the entire function of thought is to produce habits understood as 
cultural dispositions to action since to understand the meaning of a thing 
is to determine what habits it produces, for what a thing means is simply 


44 Cristina Voto 


what habits it involves. This is why it is possible to state that there is no 
distinction of meaning that does not consist of a possible practical differ- 
ence. Meaning is to be sought in the effects and performativity that shape 
our sociocultural life. Consequently, if we wish to understand the meaning- 
effects that automated facial images produce within our culture, it is worth 
starting with the habits spread. 

If we consider the global processes of broad diffusion of facial recogni- 
tion technologies that started exactly after the terrorist attack of Septem- 
ber 2001(Gates 2011) and the general resignification of the private sphere 
in favor of public security that occurred at the time — as testified by the dis- 
semination of Paul Ekman’s theories of emotion analysis and recognition‘ 
— in addition to the appearance of surveillance studies (Lyon 2007; Bauman 
and Lyon 2013; Browne 2015), today facial effects seem to differentially 
permeate the porosity of otherness. To understand these new habits, I refer 
once more to Butler: 


We may have to think of different ways that violence can happen: one 
is precisely through the production of the face, the face of Osama bin 
Laden, the face of Yasser Arafat, the face of Saddam Hussein. What has 
been done with these faces in the media? They are framed, surely, but 
they are also playing to the frame. And the result is invariably tenden- 
tious. These are media portraits that are often marshaled in the service 
of war, as if bin Laden’s face were the face of terror itself, as if Arafat 
were the face of deception, as if Hussein’s face were the face of contem- 
porary tyranny. 

(2004: 141, italics in the original) 


In this passage, we find a question of the utmost importance, which is also 
formally evidenced in the original text. It deals with the processes of the 
production of faces that are the protagonists of war and the effects — the 
context that has been played out — that those processes have on the soci- 
ocultural reception of otherness. How do technologies endow the image 
of otherness? The faces of the Other become habits when they encounter 
screens which have a reactive function and the power to arrange lights 
according to ideological gazes and discourses. The encounter with the faces 
of the war and the manner of communicating them depend on the techno- 
ideologies diffused in the media context. In this sense, those reproduced 
faces embody the discursive sociocultural processes of identifying a par- 
ticular type of othering: the enemy. 

As Butler writes, those images are framed by a general visual gram- 
mar that makes them communicative forms for the embedding of other- 
ness. The faces of Osama bin Laden, Yasser Arafat, Saddam Hussein, and 
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Afghan girls are all presented in the media as portraying a general process 
of identification. Through a formal enunciative locus, the portrait genre, 
those images set up a relationship in a figurative dialogue with the specta- 
tors. But how is this dialogue shaped and how does it function? It works 
within a visual re-assemblage at the service of war, as if these faces were 
synecdoches standing for the effects of the terrorist attack itself. 

Just from these observations, we can say that each of those facial images 
of the war is deprived of the phenomenological infinite described by 
Lévinas and recuperated by Butler; there is no precariousness of life or 
interdiction on violence, nor is there transitivity: these faces are completely 
graspable, completely framed. This condition is defined by a scopic regime 
that intertwines the facial images with the semiotic forms allowed by 
warfare: they are the whole of the war. Thanks to a precise aesthetics of 
identification that guarantees the condition of production of an effect 
of verisimilitude of the enemy, through a peculiar surface effect in those 
interfaces of the war, the infinite unspeakable message of precariousness is 
canceled. The erasure, the meta-face effect, makes those faces the surface 
of violence itself, allowing habits that intervene in the interpretation of the 
images reproduced by the media, rendering the facial artifact disconnected 
from the precariousness of the human condition. 

Following a historical path that from Lévinas’ times passes through 
the conflicts in Afghanistan up to the present, when those hostilities have 
ceased after twenty years of combat and when, meanwhile, other wars 
have started, it can be observed that the faces of war are still pervasive as 
forms of communication of otherness in our iconosphere. What is more, 
today there is the spread of automated portraying of the enemy. Facing 
this contingency, in global warfare that shows no sign of coming to a halt, 
how are we to look at the contemporary flood of automated facial images? 


3. The resemblance of the iconic face of otherness 


Throughout Western history, facial images have always been considered 
sensitive forms ready for the communication of identity. Let us just think of 
the story that comes to us from Pliny the Elder about Butades, the ancient 
sculptor who executed the first relief portrait, and his daughter Kora. Since 
his daughter had outlined on a wall the shadow of her lover before he 
left for war, Butades filled that silhouette with clay, obtaining an artifact 
capable of easing his daughter’s grief at the departure of her beloved (Nat. 
Hist., XX XV). In this chronicle, too, the face is indeed considered “a figure 
that communicates”, as stated by Butler (ibidem: XVIII), a figure on and 
thanks to which a number of significant interactions take form from prac- 
tices of identification to decodifications of both physiological and cultural 
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readings. These readings, for example, are those that we find in physiog- 
nomy, the pseudo-scientific gaze for interpreting what is invisible through 
the visible features of the face. This process of decodification has been 
enormously successful in different times and cultures, as attested by Chi- 
nese physiognomy and similar fields of study that have also been diffused 
in Arabic culture. 

If the idea that from the external look of things it is possible to under- 
stand their nature is already present in Aristotle, it is with Johann Caspar 
Lavater’s physiognomy during the second half of the eighteenth century 
that an attempt is made to free the human face from the typifying tendency 
that, in antiquity, provided a biased key to interpreting observed phenom- 
ena. From a gaze that refers to a cultural model, depending on situated 
values and ideologies, Lavater updates traditional physiognomic reasoning 
with a vision of the face as a unique and individualized window through 
which to understand human character. Within this same discipline and a 
few years later, while Lavater examines the motionless features of a face, 
Georg Christoph Lichtenberg concentrates on those signs that make the 
face changeable and express its emotional state. From fixed and genetic 
facial features, we move on to an increasingly pronounced focus on emo- 
tional impulses. Lichtenberg’s intention is to avoid the popular misuse of 
physiognomy and, in response to his warning physiognomy, expands its 
domain to encompass the dimension of temporality, an interpretation that 
opens to the world and moves away from the human being, transferring its 
attention to different types of expression. 

Since antiquity, thus, and through an always-situated reading of what 
naturalistic determinism might be, according to physiognomy the face can 
be interpreted through visible data to translate them into cultural outputs. 
In line with this approach, also thanks to the diffusion of technologies 
for the mechanic reproduction of facial images, such as the chair designed 
by Lavater for taking silhouettes and latterly photography, the face has 
undergone a progressive disembodiment leading to the diffusion of forms 
of codification that circulate independently from physical bodies and that 
can be codified through an equally disembodied form. This trajectory is 
constantly being updated and leads to what we can now think of in terms 
of an algorithmic physiognomy. In all these efforts to decompose and 
recompose the face we can recognize attempts to re-create a sense of iden- 
tity. However, the progressive disembodiment that digital society confronts 
us with seems to mark a point of no return in the relationship between 
the face and its bearer: a progressive massification of data such that the 
decomposition and re-composition operations to which the physiognomic 
tradition had accustomed us cannot but be carried out by an intelligence 
capable of automatizing the processing of such a large mass of data that it 
escapes human cognition. 
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A turning point in the conception of our relationship with the face as 
a communicative form has been the development of cities and the dense 
network of mass media that is to be found in every metropolis. The urban 
dimension and the intricate social network that city fabrics weave lead to 
new discourse effects regarding processes of identification and practices for 
the attachment or resistance to a given community. Two good examples for 
grasping the status of this process are the two face archiving methods that 
began to spread in the late nineteenth century in scientific and forensic dis- 
courses, namely, Francis Galton’s composite portrait’ and the portrait par- 
leé of the French police officer Alphonse Bertillon,’ developed to identify, 
in the context of what would later become the media society, genetic and 
criminal deviances. In this regard, it may not be inappropriate to acknowl- 
edge that the media society has produced an epoch-making impact on the 
visual representation and reproduction of identity and, therefore, of other- 
ness. This shift has resulted in the unbridled production of facial icons: the 
faces of propaganda, the faces of celebrities, the faces of the oppressors, 
and the faces of the oppressed. As suggested by Thomas Macho (1996), the 
modern facial society emerged from the diffusion of media representations 
that would take the humanized presence away from the face by detaching 
perceptive habits, eliminating local physiognomies, and affirming icons. 

The phenomenon of the dissemination of facial icons is also at the center 
of one of the chapters of A Thousand Plateaus: Capitalism and Schizo- 
phrenia (1980). In the essay, Gilles Deleuze and Félix Guattari propose 
considering the fortune of the icon of the human face in Western culture 
as a biopolitical heritage. The authors recognize in Christian Europe an 
abstract machine that projects its power onto the face: faciality. This 
machine does not function to represent a particular token of humanity 
but rather to construct a type of human. Deleuze and Guattari assert that 
certain sociocultural apparatuses need the abstract machine of faciality to 
be effective and to construct a reality through the configuration of a specific 
face-type. As they write: 


The face is produced only when the head ceases to be a part of the body, 
when it ceases to be coded by the body, when it ceases to have a multi- 
dimensional, polyvocal corporeal code — when the body, head included, 
has been decoded and has to be overcoded by something we shall call 
the Face. 

([1980b] 2005: 170, italics in the original) 


The term “overcoded,” used by the authors since The Anti-Oedipus (1972), 
indicates a second-level coding process, a codification operated by a lan- 
guage capable of assigning new expressive codes. Concerning the abstract 
machine of faciality, this overcoding operates for the benefit of a semiotics 
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of subjectivation where the polyvocal corporeal code is minimized and 
materialized in a specific biopolitical environment. This faciality codifies 
the message of the precariousness of humanity through expressive values 
that commute the vulnerability of life into a type, as in the iconic faces rep- 
resented on coins that signify the commutation of a peculiar human-type 
into a value. In this sense, according to Deleuze and Guattari, the abstract 
machine of faciality is by no means universal but is what emerges from the 
forms of expression of the type of the white man, from his development 
that originates with the iconic face of Christ, with the coding that makes it 
possible to produce all the coherent units of the face and all the rejections 
of deviation, all that is proper and improper. 

The process of iconization within Western visual culture is also at the 
core of Face and Mask: A Double History (2013) by Hans Belting. In his 
book, the result of years of investigations, the art historian questions the 
common Western approach that idealizes the authenticity of the face while 
discrediting the mask as an illusion. On the contrary, from his viewpoint, 
both the face and the mask can be understood as images that manifest 
themselves through certain artifacts which have become visible thanks to 
an iconic figuration. Belting’s research starts from the recognition of a very 
complex plot that links the face and the mask as two phenomena expressed 
in many cultural histories, such as the history of art, the history of scien- 
tific representations, and the history of media. His main hypothesis can 
be thus resumed: since antiquity, Western culture has made the human 
face into a semantic-expressive form that has assumed the social figura- 
tion of the mask. From this standpoint, Belting proposes a consideration 
of the genre of portraiture, and particularly portraits from the fifteenth to 
the seventeenth century, as well as the European mask itself. Like masks, 
portraits are characterized by a certain compactness and durability (Don- 
dero 2020a), while they raise a series of questions about the design of 
visual identity. Belting’s thesis opens up a whole series of very rich reflec- 
tions, first of all on the bio-cultural criteria of relevance and selection of 
the effects of resemblance. 

Although today we tend to regard portraits as symbolic works of art, 
they were once artifacts that fulfilled certain social functions, ranging 
from being commodities of exchange to inherited goods, from objects of 
social affirmation to heirlooms. In this sense, already from the Renaissance 
on, portraits were widespread insofar as they served a proxy function — 
namely, in circumstances where the person depicted was absent but still 
wanted to assert his or her presence. Particularly in Flemish portraiture, 
it is possible to notice the transition from the depiction of a presence to 
the artifact existence of a portable object, a painted panel, which endowed 
the subject of the portrait with a symbolic presence and related state of 
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rights or authority. As pointed out by Belting, the European portrait can 
be considered a particular type of mask since it can replace the face with 
an icon. It presents the resembling face as a sign that invites the viewer to 
communicate with it. Portraits show the necessity of the sociocultural face 
to be represented; thus, in addition to the physiognomy that revealed the 
uniqueness of the person, they revealed the codified mask with which the 
person sought to assert his or her position in the social context. 

Building upon the lessons of Macho, Deleuze, Guattari, and Belting, to 
comprehend the spread of facial images in our iconosphere, it becomes piv- 
otal to understand its iconic nature. Once more, the philosophical propos- 
als of Peirce can be of help to us. In An Elementary Account of the Logic 
of Relatives, Peirce writes (1886): 


The icon represents its object by virtue of resembling it. It thus depends 
upon simple feeling. Mental association has nothing to do with it. The 
icon has no generality, because it does not analyze the character it 
exhibits. There is thus no more generality in the icon than there is in its 
object. Nor has the icon anything to do with the sense of contact with 
the world, nor with the actual existence of its object. It is a mere dream. 
Icons comprehend all pictures, imitations, diagrams, and examples. 
(MS 585: 380) 


According to Peirce, the icon entails a relationship with a referent that may 
very well not exist. This relationship is not general but concerns a precise 
object because it does not pass through conceptual meanings. The repre- 
sentation of a human face ina picture, for example, will have a certain type 
of hair or a particular nose. These objects are generated by the iconic signs 
that activate them: even when we know that an image is impossible, as in 
the case of pareidolia,’ we can still see it as a representation based on the 
functioning of our perceptions. Neurophysiological and cognitive research 
suggests, in fact, that humans are biologically predisposed to recognizing 
visual matrices with meaning in reality and, what is more, to believing 
that they emanate an intentional agency. Pareidolia confirms this perspec- 
tive: our neurophysiology drives us to recognize communicative images so 
insistently that we sometimes identify them even though these images do 
not result from any intentionality. 

Within this perspective we can say, as Umberto Eco affirms in A Theory 
of Semiotics (1976), that the iconic sign does not have the same phys- 
ical properties as the referent but stimulates a perceptual structure like 
that which would be stimulated by the referent. Thus, recovering Peirce’s 
proposals through Eco’s lesson implies being able, firstly, to identify what 
resemblance the icon stimulates and, secondly, to analyze how these iconic 
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stimuli operate in the production of habits. As in the faces of war observed 
by Butler, the facial icons prompt synecdochic forms of communication 
that intervene directly in the perception of otherness: those iconic faces are 
the war itself. 

From the conflict in Afghanistan to the present day, artificial intelligence 
systems have played an increasingly prominent role in every domain of our 
lives. In this respect, there is a thin red line connecting technological devel- 
opments, particularly those related to the visual realm, with the demands 
of war. As I write, after all, the media are announcing the news that the 
use of facial recognition systems is ongoing in the conflict between Russia 
and Ukraine, a novelty that makes me ask: what sense of otherness can be 
communicated in those facial images? 


4. Interpreting the para-faciality of otherness 


Every epoch disposes of several apparatuses that configure ways of seeing 
capable of shaping the perception of otherness. For example, the works 
of Susan Sontag (1977), Teresa de Lauretis (1987), and Jonathan Crary 
(1990), among many others, have exhaustively analyzed the specificity of 
the optic apparatus in modeling modern and postmodern society. Today, 
in times when the progressive reconstruction of reality based on a binary 
standardization is re-ontologizing our world, new habits arise. The circula- 
tion and diffusion of technologies like software for facial recognition, the 
creation and establishment of digital portraitures such as visual filters in 
social networking, and the diffusion of dematerialized approaches to stor- 
age such as visual big data or the act of scrolling in dating apps, change 
the ways of performing, and thus of understanding, the processes of iden- 
tification. A proposal for framing the complexity of the phenomenon is to 
imagine a genealogical trajectory that, from the Renaissance development 
of geometrical perspective as a mathematical eye based on a statistical code 
that translates the complexity of the world into discrete elements (Mal- 
donado 1974; Friedberg 2004), leads up to the present. We can identify 
a watershed in this genealogy in the diffusion of the mechanical model 
(Maria Tortajada and Francois Albera 2010) between the seventeenth and 
eighteenth centuries, a sociocultural process that opened a series of propo- 
sitions on the models of apprehension, particularly according to the con- 
cept of division into discrete units that can be combined. The legacy of this 
model is still present in our contemporary media ecology, as a meaning- 
effect in the discourses and practices that cross it. Nevertheless, this is an 
environment where the intelligibility of the modes of information, technol- 
ogies, and codes of communication is driven by both human and artificial 
agencies, an ecology where the units to be combined are both flesh and bits. 
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At the beginning of the twenty-first century, John Johnston (1999: 27) 
analyzes the contemporary media ecology in terms of machinic vision": 
“not only an environment of interacting machines and human-machine sys- 
tems but a field of decoded perception that, whether or not produced by or 
issuing from these machines, assume their full intelligibility only in relation 
to them”. Within this same perspective, Paul Virilio has referred to it as the 
vision machine, that is, “the possibility of achieving sightless vision” (Virilio 
1994: 59), a vision that recognizes shapes and combines them through syn- 
thetic and discrete data. In more recent times, Trevor Paglen has described 
the artificial visuality diffuse in our media ecology in terms of “invisible 
images” (2016), referring to the flow of visual data produced by machines 
and used within human-machine interaction. This is what the video artist 
Harun Farocki, in his series Eye/Machine I-III (2001-2003), named opera- 
tional images: images that do not resemble or depict a referent but are part 
of complex processes such as detecting, identifying, visualizing, tracking, 
navigating, and so on. In addition, as a sorrowful bench test of their effi- 
cacy in communicating the sense of otherness, we can see how operational 
images are increasingly central in warfare, for example, in the functioning 
of military devices such as drones. Unlike the images of precariousness, on 
their surface, human precarity is blurred and, therefore, so is the sense of 
vulnerability. There seems to be no room for empathy in the operational 
image because there is no room for the communication of humanity, only 
efficiency and the fulfillment of tasks. We can signal a paradigmatic shift 
defining the phenomenology of otherness within these images: they modify 
their proxy function, being no longer linked to the impression of resem- 
blance but to the simulation of the quantity of data stored in them. This 
shift is part of a wider biopolitical process that interweaves the contempo- 
rary diffusion of automated images with specific rhetorics. 

We can identify in the artificial way of seeing that takes possession of the 
embodied dimension of the face and turns it into an intelligible tool for rec- 
ognition, an omnipresent and omni-significant para-faciality, an inchoative 
process of identification always ready to be processed and fulfilled. From 
this standpoint, it is possible to affirm that automated facial images allow 
for the detection and identification of recurring para-facial surface effects, 
digital matrices that contain the expected metrics of facial features. They 
result from the interface effect of the artificial face concerning the required 
data necessary to model and simulate what can be recognized as human. 
These para-facialities emerge from algorithmic gestures that designate a 
technology that no longer reflects a techné but becomes an executing act- 
ant capable of revealing identity according to a differential meta-face effect. 

These gestures of artificial revelation are part of a wider algorithmizing 
process, a para-modelized procedure whereby artificial intelligence systems 
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not only govern the logic of computing but, more generally, have become 
the agents that model and reveal, by simulating, our reality. Biology, genet- 
ics, engineering, forensics, and many more areas of our lives today use 
algorithms that perform evolution, growth, adaptation, and change of data 
while they provide “a body that can be read and a body that is of use in 
virtue of its ability to produce information” (Lee-Morrison 2019: 46). At 
the same time, these algorithms teach other algorithms how to operate: 
they have ceased to be constative instructions and have become entities 
that perform, insofar as they select, evaluate, and transform; they produce 
ways of life and, of course, ways of seeing otherness: 


Algorithms learn: they adapt, adjust and evolve their behavior accord- 
ing to a qualitative synthesis of vast quantities of data. Their performa- 
tive activity is afforded by their capacity to compress large quantities of 
information and thus transform outputs into new inputs, involving a 
new synthesis of reasoning and calculation. Here data do not have to fit 
categories but are redefinable in the manner in which algorithms gener- 
ate possible rules, causes and facts where these are missing. 

(Parisi 2019: 94) 


Artificial intelligence’s algorithms, thus, are no longer just simulators of 
data dynamics since they have acquired a new status that is not related 
to the preexistence of biophysical matter. They do not just represent our 
reality by capturing, detecting, or recognizing objects but are perform- 
ing entities that expose the inconsistency of the proliferation of increas- 
ingly random and biased data within our societies. In this regard, the 
performative functioning of algorithms in designing facial images reveals 
how the degree of prehension proper to this artificial revelation has come 
to characterize the representation of otherness. Rephrasing a common 
cliché, it is possible to affirm that bias is neither a bug nor a feature but 
is a meaning effect. 

Within this framework, the current spread of digital para-facialities rede- 
fines certain practices that have characterized Western societies since time 
immemorial, such as those of the recognition and analysis of a face. The 
codification and, thus, the decodification of the face are historically deter- 
mined — to this day and in most cultures — by precise aesthetic, normative 
models, and biopolitical conditions. It should therefore come as no sur- 
prise that nonhuman-produced facial images have always been recognized 
as possessing a special aura capable of attributing extraordinary powers; 
they are images endowed with an absolute authenticity that determines an 
unmediated interpretation. This is why long-standing traditions concern- 
ing facial images produced by nonhuman agencies, such as the so-called 
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acheiropoieta images which intersect the history of visual cultures from 
East to West and North to South, can be useful for framing our topic: 


Since the face is so central in human behavior, facial images that are 
considered as produced by a non-human agency receive a special aura 
throughout history and cultures, as if they were endowed with extraordi- 
nary powers. Furthermore, since in many societies the face is read as the 
most important manifestation of interiority, “non-man-made” images 
of faces are attributed a status of authenticity and earnestness, as if they 
were the sincerest expression of some otherwise invisible agencies. 
(Leone 2021b “Prefazione”: 17) 


This auratic understanding of automated facial images continues to this 
day, generating a particular process of legitimation to which artificial intel- 
ligence has been recently subjected: an aletheic power (Sadin 2018). Today, 
facial images produced by artificial intelligence systems seem to be endowed 
with the extraordinary power not only to compare natural and artificial 
intelligence but also to attribute a status of authenticity to images based on 
an automated inference that quantifies the bodies and, thus, the sociocul- 
tural process of identification and construction of otherness. Indeed, this 
state of authenticity seems to rewrite the forms through which we relate to 
reality, the ways in which we experience the world around us and, above 
all, the rhetorics of verisimilitude. Immersed in an iconosphere where the 
immediacy of data produces a very strong sense of presence, where objects 
become hyper-real and, above all, where an aletheic impression of reality 
seems to permeate everything by means of a hyper-stimulated visual regime, 
which images are to be believed? And above all, which facial images will 
still be able to communicate the precariousness of otherness? 

To answer this question, I propose in what follows the analysis of an 
artistic presentation, the video Face Scripting: What Did the Building See? 
produced in 2011 as part of the investigations of the multidisciplinary 
group known as Forensic Architecture, based at Goldsmiths, University of 
London. 


4.1. The para-faciality of otherness and its uncertainty 


In January 2012, Mahmoud al-Mahbouh, an official belonging to Hamas — 
an acronym that stands for Islamic Resistance Movement, the political- 
religious organization whose goal is to liberate Palestine from Israeli 
occupation to establish a religious state there — was killed in a hotel in 
Dubai. A month after the killing of the official, the police of the United Arab 
Emirates published a video, still available on YouTube, where through an 
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operation of remixing and recomposing images extrapolated from video 
surveillance circuits of airports, hotels, and shopping centers, the aletheic 
agency of the artificial gaze is staged. This is a gaze that encodes a spe- 
cific way of seeing; that is, it encourages us to trace in the succession of 
images that make up the video a legible, and plausible, crime scene. This 
rhetorical effect finds its essence in the verdictive nature of the images of 
the video surveillance circuits, in their clinical precision ready to guaran- 
tee total adherence to reality thanks to the aletheic power of the facial 
recognition software used to analyze those recorded images. In just under 
thirty minutes of images, the reel released by the Dubai police attributes the 
murder of Mahmoud al-Mahbouh to agents of Mossad, that is, the intel- 
ligence agency of the State of Israel, focused on foreign operations. Once 
posted on the YouTube platform, this video became a real agent, part of the 
crime scene. And it is precisely thanks to the recognition of a widespread 
para-faciality in the spaces examined, a latent figure always ready to com- 
municate an identification process, that the police tried to identify the faces 
of the murder suspects in the crowd. 

However, the video cannot be classified as a document, as the trace 
or the evidence of a crime but as the montage of a series of possible 
scenarios and faces. In fact, during its scant 30 minutes we witness a 
mise en scene where, through the result of the intersection of a narrative 
flow between spaces and faces, between architecture and face images, 
the faces of the suspects and the suspicious become latent surfaces of 
criminality, an interface effect that reduces the meta-effect of those faces 
to the verification of a crime scene. They are faces, after all, that are 
denied any possibility of grasping human infinity because of the way 
the mechanisms of the gaze are positioned: video surveillance cameras 
deny the possibility of looking back, emphasizing strongly asymmetrical 
power relations. 

In a sort of digital update of the portrait parlé techniques by Bertillon, 
the Emirati police extracted a whole series of data from the surface effect 
played out by the images captured by video surveillance circuits, such as 
the relation of forehead to hairline, the relation of nose width to eyes, and 
the relation of jaws to cheeks. To rewrite the crime scene, each moment of 
encounter between Mossad agents or between the agents and unidentified 
persons provides the surveillance scanning algorithm with a possible line 
of investigation. This line is disentangled from the impression left by the 
interface effect which, thanks to the material actualization of otherness, 
follows new faces in the crowd to meet other suspects. The para-faciality 
to which the media ecology of our digital contemporaneity is accustoming 
us eventually allows the inchoative opening of new avenues of investiga- 
tion, bifurcations that could continue ad infinitum, because every time the 
identified agents meet colleagues in Dubai or in any other country, other 
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agents will also be identified due to a particular meta-face effect: these faces 
stage intransitive dominant forms of communicating otherness while com- 
municating quantified data concerning what can be understood as a precise 
human-type, the suspicious. 

The collaborative video Face Scripting: What Did the Building See? co- 
made by Shumon Basar (writer, editor, and curator), Jane & Louise Wilson 
(an artistic duo engaged for years in audiovisual and installation works), 
and Eyal Weizman (founder and director of the research group Forensic 
Architecture), reflects precisely on the veridical rhetoric of para-faciality in 
the age of algorithmic reproducibility. It is part of the investigations real- 
ized by Forensic Architecture a group, founded in 2010, which develops 
its work in the field of historical, theoretical, and artistic research starting 
from a re-appropriation of contemporary forensic practices to critically 
evaluate their epistemologies, protocols, and knowledge production poli- 
cies. Forensic Architecture practice starts from a deep and sensitive under- 
standing of the two fundamental aspects of every forensic investigation: 
the field and the forum. Central to their practice is, thus, a focus on what 
we can define in terms of a material investigation, that is, research that 
seeks to bypass human testimony in favor of the material findings that can 
be brought to the forum by making the matter a political agent. As in all 
research developed by this group, it is the forensic turn on the space of the 
visible that is at the center of Face Scripting: What Did the Building See? 

At the very beginning of the video the voice-off defines the piece as “a 
narrative assembled by an algorithmic forensic based on the architecture of 
the human face”, the design which describes surface effect through the tax- 
onomies of eyes, noses, forehead, mouth, and the distance between them. 
Then it continues: “algorithms identify individuals by extracting and ana- 
lyzing landmarks from the images of the face”. Using the theoretical system 
built up throughout these pages, we can erect an analogical bridge between 
the image of the landmark and the interface effect, always primed for the 
inchoative process of identification to occur, the face being central for 
the recognition of what is suspicious. While the voice frames the narrative, 
the filmic enunciation guides us through a free indirect discourse where we 
witness the undecidability of the point of view on subjectivity: we glimpse a 
screen, the reflected image of which alerts us to the possibility of a gaze that 
is absent but nevertheless ready to grasp its contents; the revolving door 
of a hotel entrance observed at human height without, however, a body 
passing through it; a journey from the bottom to the top, almost simulat- 
ing an absent face gazing upward at the ceiling with its nose in the air, 
along what appears to be the corridor of a luxury hotel. We are faced with 
the impossibility of assigning an identity, a putting into perspective that is 
totally antithetical to the rhetoric of hyper-visibility enacted in the video 
released by the Dubai police. In the video Face Scripting: What Did the 
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Building See? it is through the material dimension, through its situated sig- 
nification that an attempt is made to make sense of — and be sensitive to — 
what has been considered a simple thing. It is the walls of large shopping 
malls and luxury hotels, the spatialities made up of corridors, and build- 
ings whose interstices are not completely reachable by the watchful gaze of 
security cameras that are the protagonists, with their affordances and usage 
programs. Restarting from the material dimension means, then, putting the 
connections between human and nonhuman, between the artificial eye and 
the embedded dimension of perception, back in the center of the frame. 
It means bringing back into the limelight the efficacy of models, simula- 
tive and interpretive, made of flesh and bits, those which hopefully may 
allow for some uncertainty in the process of subjectification with which to 
counteract — or at least assign an out-of-frame to — the inchoative nature of 
digital para-faciality. 


5. Conclusions 


Throughout these pages, I have aimed to encompass how, in the contem- 
porary iconosphere, facial images do things while taking into consideration 
differential perspectives of communicating otherness: from the circulation 
and use of facial icons to the mutability of facial formats and the biopoliti- 
cal implications of the storage of large volumes of data regarding faces. 
Furthermore, observing the implementation of technologies capable of 
automating perceptual models, I have framed a differential understanding 
of contemporary ways of seeing and communicating otherness in terms 
of para-faciality. All these phenomena raise new questions concerning the 
processes of quantification used for the representation and recognition of 
otherness in a cultural context in which an increasing number of images 
are produced and consumed not only by humans but also by machines. 
To think that otherness can happen through the production of automated 
facial images allows a material turn to be applied to the medium itself. In 
this regard, if the term “media” can be used to refer to an established and 
institutionalized communicational environment, then conversely, by using 
the word “medium” we can refer to the net of material components that 
produce the uncertainty of images. On this matter, I have referred to the 
artificial representation of the human face as a sociocultural set of visual 
techno-ideologies produced by always-situated facial effects that allow dif- 
ferential forms of communicating otherness. This switch from media to 
medium has also permitted considering the scopic regime that produces 
and enables facial images, the dynamics associated with the mechanisms of 
visibility and knowability of social bodies and, of course, of social faces. 
At the same time, my inquiry into the scopic regime that defines portraiture 
has focused on the analytical effectiveness of visual and semiotics studies as 
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productive theoretical instruments for reading the present and for develop- 
ing strategies useful for untangling cultural complexity, even in contempo- 
rary warfare. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 

2 “The first comprises the qualities of phenomena, such as red, bitter, tedious, 
hard, heartrending, noble; and there are doubtless manifold varieties utterly 
unknown to us. [. . .] It is sufficient that wherever there is a phenomenon there 
is a quality; so that it might almost seem that there is nothing else in phenom- 
ena. The qualities merge into one another. They have no perfect identities, but 
only likenesses, or partial identities. Some of them, such as the colors and the 
musical sounds, form well-understood systems. Probably, were our experience 
of them not so fragmentary, there would be no abrupt demarcations between 
them, at all. Still, each one is what it is in itself without help from the others. 
They are single but partial determinations.” 

3 “The second category of elements of phenomena comprises the actual facts. 
The qualities, in so far as they are general, are somewhat vague and potential. 
But an occurrence is perfectly individual. It happens here and now. A perma- 
nent fact is less purely individual; yet so far as it is actual, its permanence and 
generality only consist in its being there at every individual instant. Qualities 
are concerned in facts but they do not make up facts. Facts also concern sub- 
jects which are material substances. We do not see them as we see qualities, 
that is, they are not in the very potentiality and essence of sense. But we feel 
facts resist our will. That is why facts are proverbially called brutal. Now mere 
qualities do not resist. It is the matter that resists.” 

4 “The third category of elements of phenomena consists of what we call laws 
when we contemplate them from the outside only, but which when we see both 
sides of the shield we call thoughts. Thoughts are neither qualities nor facts. 
They are not qualities because they can be produced and grow, while a quality 
is eternal, independent of time and of any realization. Besides, thoughts may 
have reasons, and indeed, must have some reasons, good or bad. But to ask why 
a quality is as it is, why red is red and not green, would be lunacy. If red were 
green it would not be red; that is all. And any semblance of sanity the question 
may have is due to its being not exactly a question about quality, but about the 
relation between two qualities, though even this is absurd. A thought then is 
not a quality. No more is it a fact. For a thought is general. I had it. I imparted 
it to you. It is general on that side. It is also general in referring to all possible 
things, and not merely to those which happen to exist.” 

5 “Let us use the word habit . . . not in its narrower, and more proper sense, in 
which it is opposed to a natural disposition (for the term acquired habit will 
perfectly express that narrower sense), but in its wider and perhaps still more 
usual sense, in which it denotes such a specialization, original or acquired, of 
the nature of a man, or an animal, or a vine, or a crystallizable chemical sub- 
stance, or anything else, that he or it will behave, or always tend to behave, ina 
way describable in general terms upon every occasion (or upon a considerable 
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proportion of the occasions) that may present itself of a generally describable 
character.” 

6 After September 2001, Paul Ekman’s theories became very popular in US cul- 
ture and in other disciplinary fields besides psychology. Ekman’s laboratory 
approach promised to make others’ emotions truthful and unambiguous re- 
gardless of the awareness of the person who was experiencing those emotions. 
In this regard, Jan Plamper (2015) indicates ideological parallelism between 
Ekman’s approach and the logic of political national security that spread after 
the fall of the Twin Towers. In the laboratory, the certainty of the analysis is 
the result of the specific epistemology of experimental psychology which is the 
reduction of the reality of certain dependent and/or independent variables. This 
reduction cannot be reproduced in the real world. 

7 Developed in the 1880s, the composite portrait creates the criminal type from 
the statistical averages of the anthropometry of the face according to essential- 
ist biometrics. The starting point of the composite photograph is a repetition 
of shooting and assembling acts that function as a promise against the vari- 
ance of human conditions. Galton believed that he had translated the Gaussian 
curve into the pictorial image which wires a human face (cfr. Sekula 1986; Lee- 
Morrison 2019). 

8 For Bertillon, the mastery of the criminal body necessitated a massive campaign 
of inscription, a transformation of the faces into texts that pared verbal descrip- 
tion down to a denotative shorthand, which was then linked to a numerical 
series. Firstly, he combined photographic portraiture, anthropometric descrip- 
tion, and highly standardized and abbreviated written notes on a single fiche. 
Secondly, he organized these cards within a comprehensive, statistically based 
filing system. Bertillon would go on to hone his system of physical description 
into a morphological vocabulary to describe the variability of human features, 
as well as a system of abbreviations to render that vocabulary communicable 
by telegraph. 

9 Pareidolia is a perceptual illusion that allows the recognition of figures en- 
dowed with meaning, faces in most cases, in plastic configurations where this 
meaning has not been introduced by any human intentionality. From a biologi- 
cal point of view, pareidolia is the result of a long perceptual process linked to 
the survival of the human species through which the recognition of hostile faces 
in the environment is fundamental. 

10 Johnston borrowed the term “machinic” from A Thousand Plateaus. In their 
essays, Deleuze and Guattari oppose the machinic on the one hand to the me- 
chanical, which applies to the machine as a functional unity of discrete but 
homogeneous parts and on the other to the organic, which applies to the or- 
ganism as a hierarchical organization of biological organs. Where bodies and 
machines enter into machinic relationships, that is, become parts of an assem- 
blage, Deleuze and Guattari distinguish two opposing processes: the decoding 
or deterritorialization where a functional equilibrium gives way to movements 
of change and becoming, and the opposite side of the assemblage where there 
are processes of stratification or reterritorialization. 
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A dramaturgy in digital 
environments! 


Enzo D’Armenio 


1. Introduction 


In this chapter, we seek to analyze the construction of meaning performed 
by new social actors — especially the so-called influencers — on online 
social networks. In particular, we will attempt to characterize the role 
of the face in their images by the way they develop peculiar narratives 
through photographic posts and video productions.” We will then attempt 
to define a dramaturgy of the face and two genres of images that exploit 
it on social networks: in particular, the new portrait genres (selfies and 
identity-related images) and video performances (live streaming and You- 
Tube footage). 

This focus on the face has a twofold motivation. On the one hand, the 
role of visual identities seems to have been strengthened on social networks, 
allowing new personalities and new professions to emerge mainly owing 
to their physical personality and character. The action of influencers, who 
are able to build communities of followers around themselves, redefines 
the concept of entertainment carried out by more traditional media such as 
television and cinema. On the other hand, the meaning-generating action 
of the face departs from the canons of fiction and traditional portraiture, 
to the point of revealing new semiotic qualities in which social identity and 
personal identity are fused in peculiar mimicry performances. 

However, the role of the face must be framed in relation to the dynamics 
already in place on social networks. The actions of influencers show how 
identities are at the center of multiple social mediations. The way in which 
they build quantitative value through large communities of followers paves 
the way for other types of value transformation: financial value, through 
the monetization of visits and subscriptions, but also reputational value 
with appearances in traditional media such as television and print. Framing 
the context of the action of these actors is therefore fundamental to under- 
standing their particular identity narrations and the role that the face plays 
in the images they produce. 
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For this reason, we will proceed in three stages. Firstly, we will attempt 
to characterize the pressures exerted on identities within social networks, 
paying attention to the peculiar overlap between the experiential and algo- 
rithmic dimensions that characterize digital networks. A brief analysis of 
the semiotic value of the acts of liking and following, together with striking 
examples of influencers, will allow us to provide a semiotic background 
for analyzing their actions. A reinterpretation of Paul Ricoeur’s theory of 
identity will allow us to locate the peculiar narrative structure that governs 
the construction of identities on social networks. 

In the second part, we will focus on the role of the face, starting from 
Hans Belting’s considerations contained in his Face and Mask: A Double 
History (2017). We will attempt to identify the qualities that make of the 
face not only a semiotic device but also one of the most effective ones, all 
the while relating its characteristics to those of another expressive medium: 
images. Finally, we will present two fields of analysis of social networks 
across which is deployed what we call a dramaturgy of the face, although, 
in light of Belting’s hypotheses, we could rather call it a dramaturgy of face 
masks. On the one hand, we will compare the principles of artistic portrai- 
ture with identity images on social networks. On the other hand, we will 
discuss the transformation of the actor into a fictional character in order to 
identify the specificities of the so-called influencers. This dual comparison 
will be an opportunity to articulate, from a semiotic point of view, the 
theoretical frameworks proposed respectively by Ricoeur and Belting. 


2. The economy of attention and appreciation on social 
networks: toward an experiential regression 


The case of the so-called influencers allows us to frame the dynamics of 
interaction on social networks because they push their functioning to 
the limit, exhibiting the mechanisms of construction and transformation 
of value. According to a distinction recently proposed by Lev Manovich 
(2017) pertaining to images on Instagram, it is possible to define the action 
of influencers as a competitive practice, in which the shots produced and 
shared are aimed at gaining maximum appreciation and visibility, as 
opposed to casual photos, which follow the pattern of “home photos” 
(Chalfen 1987) and which are therefore aimed at the niches formed by 
family and acquaintances.* 

According to our hypothesis, the activity of the influencers shows in an 
emblematic way the emergence of a new economy of values. The notion of 
economy to which we refer is inspired by semiotician Jean-Francois Bor- 
dron, according to whom economies are the social systems that regulate 
the emergence of all forms of valorization: “The economy refers first and 
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foremost to the order that underlies the possibility of values and their pos- 
sible circulation” (Bordron 2010: 37, our translation). In short, an econ- 
omy would be “a set of instances, presupposing an operation of sharing 
which institutes them in their differences and, by that, in their possible 
relations. .. . To each of these instances corresponds a more or less privi- 
leged origin of value” (Bordron 2010: 38, our translation). In this context, 
the instances governing linguistic and communicative values are part of a 
more articulated system, intertwining with the instances relating to other 
types of value. 

Let us first consider the formation of monetary value while focusing on 
the case of MrBeast, the YouTube channel of an influencer who publishes 
video montages in which he makes an aggressive use of money. In one set 
of videos, in particular, he visits the video channels of little-known stream- 
ers and sends them donations of thousands of dollars for futile reasons.* 
He targets young gamers engaged in live gaming sessions of Fortnite, a 
competitive video game whose popularity has made it a mainstream phe- 
nomenon, and makes large donations as long as these streamers admit 
that they are children. The resulting video is a montage of the reactions of 
extreme surprise and jubilation at the thousands of dollars received. 

This example clearly shows the emergence of a particular economic 
logic: the influencer can afford to spend his money aggressively because the 
resulting video attracts enough views and subscriptions to his channel to 
justify it. This video has garnered over 38 million views and six hundred 
thousand likes, while the channel has a total of over 37 million subscribers. 
With these numbers, YouTube commercials guarantee a significant return, 
not to mention the numerous sponsorships from third parties. This creates 
a virtuous circle — from a financial point of view — in which the influencer 
can donate money that will then be repaid through the monetization of the 
attention and appreciation of users, measured on the basis of likes, sub- 
scriptions, and views. However, this example also allows us to question the 
appreciation expressed by the acts of liking, following, and sharing at the 
center of this economy: what kind of semiotic acts are they? And how do 
they contribute to sustaining the particular economy of values that governs 
social networks? 

First of all, they are spontaneous, simple semiotic acts, easy to perform 
because they are linked to a basic aesthetic reaction, which can be identified 
with what in semiotics is called the thymic category, that is, the basic affec- 
tive disposition articulated into euphoria (attraction), dysphoria (repul- 
sion), and aphoria (neutrality). Secondly, they are very flexible acts because 
they are as appropriate for expressing appreciation toward a simple pho- 
tographic shot, such as a selfie, as they are for showing support toward a 
discursive opinion on a sensitive issue. Finally, they are measurable acts 
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because they are expressed by a single person, or at least by a single profile, 
in a direct way. 

This first framework of semiotic features configures a sort of experiential 
regression in the fruition of contents, precisely because they are immediate, 
simple reactions, ideally close to a basic aesthetic liking. On the other hand, 
another important factor must be taken into account, one which concerns 
the socialization of discursive contents: the acts of liking, following, and 
sharing — as well as comments — are processed automatically by algorithms, 
which spread the contents in various ways to wider circles of collectives, 
thus proving decisive in the formation of communities. This constitutes a 
paradoxical experiential regression, in which contents are valued on the 
basis of simple acts of appreciation but are also managed by an algorithmic 
intelligence that automatically regulates their socialization. The combina- 
tion of these two factors generates an economy in which appreciation is 
certain and measurable and allows the video to be monetized even if view- 
ers do not subscribe in cash or do not express an articulated adherence 
to the contents: it is an attention and gift-based economy (Lanham 2006; 
Casilli 2011). An illuminating comparison, in this sense, is the technique 
for measuring television audiences, in which sample polls return percent- 
ages of the share of each show, often with questionable approximations. 
For instance, it is not possible to measure the degree of appreciation of a 
show, but only whether the viewer was actually tuned in to the relevant 
channel. Compared to this method of measurement, appreciation on social 
media is more certain, precisely because it is expressed in a simple way 
by users as it is perfectly measurable and linked to the actual viewing of 
content.’ 


3. Paul Ricoeur’s theory of identity applied to social networks 


In light of this brief semiotic exploration of social networks and of the 
peculiar economy of attention and appreciation configured by acts of lik- 
ing, it is possible to isolate two major trends that impact the formation 
and the management of identity on social networks. On the one hand, we 
have observed a sort of experiential regression in the interactions due to 
the abundance of simple images which are often adherent to lived experi- 
ence, as in the case of selfies and identity-related images, as well as in the 
acts of liking, following, and sharing, which are the aims of the competitive 
practices pursued by influencers. 

On the other hand, an outsourced management of identity behavior 
emerges, as it is increasingly delegated to algorithms. This is the way Face- 
book recommends circles of friends based on the ones we already have or 
how Netflix offers us personalized audiovisual palimpsests based on our 
previous viewings and preferences, and how Tinder and Grindr present us 
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the profiles of potential partners, modeling our aesthetic dispositions and 
preferences expressed by the faces we liked in the past, (Finn 2017). Our 
hypothesis is that images on social networks always express a negotia- 
tion strategy for dealing with these two pressures, even when it results in 
an ineffective solution which lets itself be pulled toward one pole or the 
other. In order to fully understand this dynamic pertaining to identity and 
to elaborate a theoretical framework capable of framing the fundamental 
dimensions of identity-related images on social networks, we may propose 
a reinterpretation of Paul Ricoeur’s theory of identity, exhibited in the now 
classic volume Oneself as Another (1992). According to Ricoeur, identity 
is made up of two interrelated poles, which form a dynamic unity. On the 
one hand, we have idem-identity as the permanence of human traits such as 
dispositions, habits, and more generally what is related to the permanence 
of character and body. In this work, Ricoeur refers to the character as 
“the set of distinctive marks which permit the reidentification of a human 
individual as being the same” (Ricoeur 1992: 119) and as “[designating] 
the set of lasting dispositions by which a person is recognized” (Ricoeur 
1992: 121). The habits and dispositions of one’s character, for example, 
are described in terms of “a history in which sedimentation tends to cover 
over the innovation which preceded it” (ibidem). In short, the definition of 
character and idem-identity tends to qualify the features of identity linked 
to permanence, even if they are the result of changes due to habits assumed 
over time. Becoming static, a habit of this kind must be understood as a 
trait: “a character trait, a distinctive sign by which a person is recognized, 
reidentified as the same — character being nothing other than the set of 
these distinctive signs” (ibidem). 

On the other hand, the ipse-identity is conceived of as the maintenance 
of the self in the long run, resulting from a management of behavior: “the 
selfhood of the self [implies] a form of permanence in time which is not 
reducible to the determination of a substratum” (Ricoeur 1992: 118). It 
is formed through the choice of heroic role models, values, and ethical 
principles to which one remains faithful over time: a practical and ethi- 
cal orientation resulting from the evaluative choices about one’s own 
identity. It is therefore possible to distinguish the fundamental difference, 
together with the necessary articulation, between the tendency toward the 
permanence of the idem-identity and the tendency toward innovation and 
self-preservation of the ipse-identity: in the choice of ethical principles, 
values, and heroic figures, “an element of loyalty is thus incorporated into 
character and makes it turn toward fidelity, hence toward maintaining the 
self” (Ricoeur 1992: 121). This innovative tendency of the ipse-identity 
will then be sedimented and internalized “in such a way that the person is 
recognized in these dispositions, which may be called evaluative” (Ricoeur 
1992: 122). The ipse-identity, in short, is not linked to permanence of 
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character but to fidelity to the given word. The example of the promise 
allows Ricoeur to further specify this second axis of identity: “even if my 
desire were to change, even if I were to change my opinion or my incli- 
nation, ‘I will hold firm’” (Ricoeur 1992: 124). In other words, it is the 
“properly ethical justification of the promise” (ibidem) that exemplifies the 
fidelity to the given word, to the figures chosen as guides, to the durable 
values elected for the self. 

Between these two poles of identity, especially as a consequence of the 
intrinsically temporal character of life, there is a persistent “interval of 
sense” (ibidem) that must be filled to build a durable identity. In order 
to manage these two polarities and constitute an identity that can resist 
throughout a lifetime, it is necessary, according to Ricoeur, to build an 
identity narration capable of linking them. The narrative intelligence of a 
story would be able to constitute a revisable and accident-proof identity 
unit, harmonizing the two poles through successive revisions — the perma- 
nence and resistance of the idem-identity, the fidelity to the given word, 
and the variation of the ipse-identity. The meaning of the narrative unity 
of a life, in fact, 


must be seen as an unstable mixture of fabulation and actual experience. 
It is precisely because of the elusive character of real life that we need the 
help of fiction to organize life retrospectively, after the fact, prepared to 
take as provisional and open to revision any figure of emplotment bor- 
rowed from fiction. 

(Ricoeur 1992: 162) 


According to Ricoeur, the models of literary fiction are the most suitable to 
provide a repertoire of solutions to manage the complex tension between 
the two poles of identity, as they constitute a vast reflexive laboratory for 
“the application of fiction to life” (Ricoeur 1992: 161). It is through an 
imaginative and narrative appropriation that the course of a life, with all 
its accidents and choices, can be built into a meaningful form and hold up 
in the long run. This complex theory, which we present here in its essential 
characteristics, is summarized in Table 4.1. 

The structure of the narrative identity elaborated by Ricoeur’s theory 
seems of great value for our purpose here, but a series of adaptations must 
be made in view of its application to the sphere of social networks. First 
of all, it is possible to place the experiential regression due to the diffusion 
of simple acts such as like, follow, and share, as well as images adhering 
to experience and to the body, in the sphere of influence of idem-identity. 
These acts are linked to our disposition, to what we consider to be our 
character traits as sedimented over time. On the contrary, the modeling of 
user behavior by algorithms and the resulting recommendations concern 
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Table 4.1 Jean-Marie FLOCH. A schematization of Ricoeur’s theory of identity 
(Floch 2000: 32) 


Narrative identity 


Character Truth toward others [parole tenue] or 
preserving oneself [maintien de soi] 
Perpetuation Perseverance 
Continuation Consistency 
Covering the ipse Freeing the ipse in relation to 
by sameness sameness 
Sedimentation Innovation 


Table 4.2 Our reinterpretation of Paul Ricoeur’s identity theory 


Identity on social Idem-identity Ipse-identity 
media 
The experiential pressure The modeling of identity 

expressed by like, accomplished by 
follow, share acts, as algorithms and condensed 
well as the diffusion into dynamic profiles of 
of body and face- behavior; the delegation 
related images (selfies, of the process of 
subjective shots, direct preserving oneself to an 
feed videos) algorithmic “intelligence” 


Identity narration: new forms of visual narration, which 
adopt, deviate, betray, and serialize the canons of 
portraiture and other identity-related genres (face reveals, 
reaction videos, and so on) 


a delegation of the ipse-identity — the maintenance of identity over time 
and the fidelity to oneself — to an algorithmic “intelligence”. Analyzing our 
behavior and preferences, the algorithms model them and program a set of 
parameters for future propositions on the assumption of a compatibility of 
preferences and of an overall continuity of practices. 

In accordance with Ricoeur’s theory, we think that, in order to manage 
the tension between these two pressures — experiential and algorithmic — it is 
necessary to build an identity narration capable of mediating and harmoniz- 
ing them in a suitable and durable form. However, the models of this nar- 
rative mediation are necessarily different from those described by Ricoeur. 
They are no longer literary models because the proliferation of photos and 
videos on social media requires us to take into account the specificities of the 
language of images, their figurative and plastic devices, and their narrative 
structures, which replace the imaginative laboratory of literature. Accord- 
ing to this framework, identity narrations are often expressed through sim- 
ple images or performances in the case of videos, in which the idem-identity 


66 Enzo D’Armenio 


is exhibited according to solutions that can structure it temporally. In short, 
it is only by fully understanding the plastic, figurative, and generic charac- 
teristics of the identity-related images that we may understand the narrative 
pressure to which identity is subjected on social networks. 

The mediation between idem-identity and ipse-identity is often skewed 
toward one of the two poles, sometimes through a reduction of identity to 
a simple serial exhibition of one’s own face and body or sometimes even to 
a partial and stereotyped version of one’s own character, that is, a limited 
part of the idem-identity. 


4. The way images and faces compose 


It is clear that identity narration on social networks exploits the body, 
and in particular the face, in a privileged manner. This is why, in order to 
understand productions such as selfies, face reveals, and live streaming, 
we need to address a fundamental question head-on, one which concerns 
the relationship between the semiotic qualities of the face and the semiotic 
qualities of images.° In his book Face and mask: a double history (2017), 
Belting proposes a series of enlightening considerations that allow us to 
describe the face as a semiotic device. To summarize, we may say that Belt- 
ing gives a central role to the dynamism of facial mimicry, to the plastic 
appearance of the face animated over time. On the one hand, the face is 
something that exhibits moods and emotions and that connects “internal” 
feelings by exhibiting them externally. On the other hand, mimicry can be 
controlled to hide these same emotions and internal states, thus building 
a device capable of lying, as in one of the most famous and provocative 
definitions of semiotics proposed by Umberto Eco.’ 


The same face expresses truth and falsehood: at times someone can viv- 
idly reveal his “inner self” to us; at other times he conceals himself with 
an impassive face as though from behind a lifeless mask. 

(Belting 2017: 17) 


What interests us, however, is the relationship between the dynamism 
of mimicry and the production of particular expressions, which Belting 
frames by means of two metaphors: on the one hand, the face produces 
masks, for example to conceal and protect emotions and not let them tran- 
spire outward. On the other hand, and more profoundly, Belting argues 
that the face is a plastic device that composes images. “In life, expressions 
change the face we have into the face we make” (ibidem). And more in 
detail: “When the face is animated by expression, gaze, or language, it 
becomes the locus of many images. It follows that the face is not merely an 
image but also produces images” (Belting 2017: 21). 
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We would like to point out a strong correspondence with the expres- 
sive system of images. Authors such as Jean-Francois Bordron (2011) have 
rightly pointed out that the meaning system of images cannot be traced 
back to verbal predication, precisely because images signify in a completely 
different way: like faces, images do not predicate but rather compose. Cer- 
tainly, we can immediately identify a similar syntax in the construction 
of meaning by faces and images: for example, the fact that they do not 
stimulate linear but rather “tabular” readings in which centrifugal and cen- 
tripetal tensions, progressive transformations, and opposing force flows 
contribute to the production of meaning. 

Another interesting aspect of this relationship concerns the central- 
ity of perceptual mechanisms in the semiotization of images and faces. 
It is a matter of relying on a perceptual semiosis, one based on Eco’s 
primary iconism (Eco 2000), for example, rather than on an articulated 
meaning that relies on linguistic categories. In both images and faces, 
“recognition” is a fundamental semiotic mechanism, whether it is a mat- 
ter of recognizing a face or of recognizing an emotion as expressed by a 
particular configuration. In both images and faces, it is the recognition 
of such or such specific trait, for instance, a particular configuration of 
fear or a particular shade of yellow that counts, and not the categori- 
cal generality of the concept of “yellow” or “fear”. We can describe a 
face verbally, even in minute detail, but what we lose is precisely the 
fundamental identity core that allows us to know and recognize it for its 
compositional qualities, and the same applies to images. Without going 
into details pertaining to perceptual mechanisms, what this parallelism 
allows us to do is to identify the way in which image configurations and 
face configurations are stabilizing on social media compared to past rep- 
resentations of the face. 


5. The face in still images: portraits, selfies, 
and identity-related images 


First of all, as far as still images are concerned, we need to recall how 
the classical genre of the portrait resolves the tension between the com- 
positional qualities of the face and the compositional qualities of images. 
Belting rightly tells us that the portrait stands in relation to the face as an 
externalized and objectified mask on a support, which makes it a durable, 
transportable object but which inevitably erases the mobile, temporal, and 
dynamic character of mimicry. Art history and semiotics have formalized 
the conventions of the portrait by identifying those compositional tensions 
that operate between visual and facial configurations, tensions such as in 
the relationship between the figure and the background, which must allow 
the face to stand out, the tensions in the gaze directed at the spectator, who 
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configures a form of dialogue and presence, and the compactness of the 
figure represented. 

However, two aspects are of particular interest to us: the first concerns 
the way in which the visual mask relates to the identity of the person rep- 
resented. Anne Beyaert-Geslin (2017) and Maria Giulia Dondero (2020), 
who have carried out some of the most important semiotic studies on the 
subject, insist on the iconization of the character of the individual, which 
is summed up, condensed, and best presented in the portrait. It is therefore 
the production of a unique and significant image, built upon the intensity 
of the centripetal relationship that draws the viewer’s gaze toward the por- 
trayed subject. For this reason, the portrait is characterized by the absence 
of movement and by a neutral, posed expression. The portrait must be 
capable of condensing the subject’s past, present, and future, capturing his 
or her identity and destiny in a single image. For this reason, in classical 
pictorial portraits, the objects and the background often build a narration 
capable of harmonizing the visual aspects of the character, which are the 
traits of the idem-identity expressed by the face with visual elements per- 
taining to the subject’s conduct and thematic role, that is, the ipse-identity 
as may be expressed by effigies, professional or rank objects. In short, a 
classical portrait is a unique image, turned toward the future, in accord- 
ance with a memorial temporality, capable of iconizing a narration identity 
within a fixed image. If we compare this model to the selfie, and more 
generally to identity-related images that circulate on social networks, some 
macroscopic similarities, but also major differences emerge. First of all, 
with regard to the selfie, it should be noted that it is a kind of image that 
could be defined as belonging to the subgenre of the self-portrait. 

The first difference between classical portraits and social network images 
relates to the technical expertise required to produce them: the classical 
portrait requires expert skill and a refined work of inter-semiotic transla- 
tion; on the contrary, selfies exploit the automatisms of the photographic 
medium and can be taken repeatedly on any occasion. Compared to por- 
traits, however, the major differences are linked to the temporal dimension 
and to the various identity dynamics. Social network shots are images that 
do not aim at iconizing and condensing a character within a single shot but 
rather build a repeated series of shots that pluralize identity, fragment it, 
and re-enact it in different contexts. 

The temporality of production and reception constitutes a pellicular tem- 
porality, which develops itself in close proximity to the acts of visualization 
and sharing. In other words, it is not a question of building a visual narra- 
tive that can summarize the life of an individual but of a theatricalization 
of the face in accordance with a management of the presence of the self, 
which is a presence to be reaffirmed, multiplied, and pluralized over time 
in the public sphere. With respect to Ricoeur’s considerations, we could say 
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that while the portrait aims to build a narrative elaboration of the overall 
identity (idem and ipse), social network portraits are often a simple exhibi- 
tion and multiplication of the idem-identity. The example of the Instagram 
profile of Rupi Kaur — a Canadian poet and influencer of Indian origin — 
helps us to clarify this point. Let us briefly consider her production from 
the beginning of her activities until today. 

It is possible to identify a change in style from a casual use of identity- 
related images to a competitive, planned, and effective exploitation of her 
images. In other words, we could describe the process of transformation 
from a casual user to an influencer. Her profile is characterized by a wide 
variety of deformations of the portrait genre, some of which may be seen 
in contemporary pictorial and photographic art, including turning one’s 
back to the camera, the use of veils, and atmospheric occlusions, but also 
fragmented portraits, which are typical of social networks, as noted by Lev 
Manovich (2017). 

Moreover, the image that made her famous, although not a portrait, 
makes sense precisely in relation to the abundance of portraits published 
previously. It is an image dedicated to the theme of menstruation, in which 
Kaur hides her face and shows her back in order to represent herself as a 
generic female individual.’ A process of depersonalization but also of gen- 
eralization and universalization is built in order to ideally embody every 
woman around an underrepresented theme and moment. The censorship 
carried out by Instagram and Kaur’s protest earned her the support of 
online communities and the attention of mainstream media. We cannot go 
into the details of this case here (D’Armenio 2022), but it is emblematic of 
the ability of still images on social networks to build a critical identity nar- 
ration, negotiating experiential and algorithmic pressure around a visual 
storytelling capable of mixing a visual character (linked to idem-identity) 
and ethical conducts (linked to ipse-identity) in a critical manner. 


6. Mobile masks: actors, characters, and influencers 
in video productions 


The second comparison we would like to present concerns video produc- 
tions and, more generally, those images capable of exhibiting the dynamic 
mimicry of the face. In particular, as far as the genres preceding social 
networks are concerned, it is the actor-character nucleus, in film but also 
in theatre, that constitutes the most emblematic case. As pointed out by 
Belting, the theatrical actor builds images, and more specifically masks, 
through his or her face, and builds a character thanks to mimicry skills: 


The mask is the role, but it is played with the real face. It is precisely in 
the face that the human being’s dissociation from himself shows most 
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clearly. That is to say, he can represent himself as someone else using his 
own face and, as a result, behave eccentrically. 
(Belting 2017: 49) 


In semiotic terms, we could say that it is a peculiar process of débray- 
age: starting from the overall articulation capabilities belonging to his or 
her own face and body, the actor projects a selection of emotional and 
mimicry features that constitute the identity of the fictional character. This 
transformation has always fascinated us because it demands that we take 
into account aspects that cannot be reduced to abstract narrative func- 
tions, as the body and face of an actor cannot be replaced with that of 
another actor without distorting the meaning of experience. Authors such 
as Edward Branigan (2006) speak of fictional supervenience: the way in 
which we see both the actor, for instance, Sean Connery, and the character, 
James Bond. Two aspects of this transformation strike us as important: 
firstly, the centrality of a peculiar competence of the actor, which we could 
define as a competence at mimicry. An actor with a calm temperament 
(idem-identity) may be able, through this competence, to build an anxious 
character. 

Secondly, it is necessary to stress the filtering role of the script because 
it allows us to find a peculiar articulation of the identity poles defined by 
Ricoeur: the actor certainly projects some of the character traits belonging 
to his or her identity, but these remain embedded in his or her person. The 
actor therefore lends part of his or her idem-identity dispositions to the 
fictional character. The fictional character, on the contrary, is a narrative 
identity, one already endowed with a closed path of transformation with 
values of departure and arrival. In short, the fictional character in the script 
is already a virtual mediation between the idem and ipse and is capable of 
building a unitary identity. The mimetic projection of the actor, together 
with the narrative mediation of the script, allows to build this transforma- 
tion, in which the face and the body of a person build a rich, dynamical, 
and narrative identity. We could qualify this process as the donning of a 
fictional mask by the actor, worn on the face (and body). 

We can relate the actor-character to the most representative and emblem- 
atic figure on social networks, one which is struggling to find a stable defi- 
nition, that is, the influencer. What these two figures have in common is 
that they owe their success to their public image and, if we take stream- 
ers into account, to their mimicry identity, to their dynamically exhibited 
personality, and especially, to their face. We will develop the comparison 
using an example chosen for its clarity, that of video game streamers. 

Usually, these live broadcasts are organized as follows: the video game 
performance is displayed full screen with the virtual avatar moving in the 
digital environment according to a text — in this case, the video game text, 
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which is already organized narratively and syntactically but certainly in a 
less strict manner than a movie script might be, since it allows the expres- 
sion of various styles of gameplay. On the right of the screen, we usually 
find the chat through which the audience can interact with the streamer. 
Most often, it is very short messages which are sent, mainly emoticons and 
peculiar forms of “slang”. Finally, a smaller window shows the face of the 
steamer who reacts live to the game performance and to the chat messages. 

Compared with the actor, the streamer does not project a character on 
the basis of his or her capabilities and skills in terms of mimicry. On the 
contrary, he or she attempts to extract his or her identity character without 
prior programming and preparation. The filter is not that of a closed narra- 
tion condensed into a script but a playful script, which organizes in a partly 
open way the mimic responses of fright, tension, exaltation, and so on. In 
other words, the narrative mediation provided by video game texts does 
not serve to build a narrative identity but to extract a live character identity. 
It is no coincidence that this filter belongs to the macro-genre of the game, 
a practice by definition situated somewhere between lived experience and 
cultural construction. The character which emerges from this interaction 
does not use a narrative mask but a playful socialized mask, whose purpose 
is precisely to embody its own idem-identity, making it emerge dynamically 
within a partially structured experiential interaction. Compared to the case 
of the actor, the temporal dimension should again be emphasized: while 
the actor builds a sophisticated performance, subjected among other things 
to the articulations of the theatrical or cinematographic language, in the 
case of the influencer, the performance of mimicry is strongly linked to the 
present moment, to the personality expressed “naturally”, with a visual 
direction that is limited to a fixed shot of his or her face. 

Reaction videos, face reveals and children’s game channels replicate this 
formula and show us the peculiar way in which the link between images and 
faces increasingly tends to adhere to an identity close to genuine, instinc- 
tive, and contagious reactions. It is a matter of taking a song, a trailer, or 
the execution of a cooking recipe as the basis for exhibiting, through facial 
mimicry, the visible expression of one’s reactions during a first encounter 
with a widely known cultural product.’ 


7. Conclusions 


In this article, we have proposed an analytical framework for understand- 
ing the images and performances of the face produced on social networks. 
We have first of all characterized the semiotic context of social networks 
by detecting an economy of attention and appreciation. At the heart of this 
economy are the acts of liking, following, and sharing: simple semiotic acts, 
close to experience and not very articulate from a discursive point of view, 
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but which are automatically socialized by algorithms. We have therefore 
proposed to consider the fundamental dynamics of interaction on social 
networks as an experiential regression because quantitative appreciation 
becomes the basis for the value transformations carried out by influenc- 
ers: from a quantitative value to a monetary and reputational value. We 
therefore focused on identities and in particular on identities expressed and 
negotiated through images. Through a rereading of Paul Ricoeur’s theory 
of identity, we identified the forms of narration through which identity 
images negotiate algorithmic and experiential pressure. We found visual 
narration on social networks to deform and betray the genres of portrai- 
ture and audiovisual fiction. 

This contextualization has allowed us to analyze the semiotic qualities of 
the face in relation to the semiotic qualities of images, following with the 
theoretical elaboration proposed by Hans Belting. Both of these expressive 
systems rely on a different mode of meaning than the verbal one: visual 
composition. In particular, according to Belting, faces, by blocking the 
intrinsic mobility of mimicry, produce images and masks. Similarly, facial 
images propose more or less dynamic versions of facial masks, capturing 
salient expressions or enduring performances. Finally, we proposed a com- 
parison between two kinds of artistic images and two genres that have 
become established on social networks. On the one hand, we compared 
the selfie and identity-related images on social networks with artistic self- 
portraits, and on the other, we contrasted the transformation of the actor 
into a fictional character with the performances of streamers on Twitch. 
The first comparison showed that portraits on social media are not char- 
acterized by their salience, by the meaningful presentation of a character 
in a single shot, capable of summarizing idem- and ipse-identity in a single 
image. On the contrary, they express a management of the presence of 
identity over time, resulting from a potentially daily multiplication and 
fragmentation of the self. This opposition can be summarized in the oppo- 
sition between the iconizing mask produced by portraits and the multiple 
experiential masks produced by social network images. 

In the case of video performances, we opposed the projection of a fic- 
tional character from the mimicry skills of an actor, capable of building a 
narrative mediation between idem and ipse with the performances of video 
streamers. Streamers set up an open and dynamic situation in order to 
stage their own personal and mimicry character in a manner that is as close 
as possible to actual experience (idem-identity). The fictional mask of the 
actor is in such case opposed to the socialized playful mask of the streamer. 
Overall, as we have seen with the example of Rupi Kaur, there are rarer 
cases of visual construction of an identity that can harmonize the idem and 
the ipse. This analysis allows us to identify a further trend: a narrowing of 
the strategies built on the peculiarities of the languages of images, of their 
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compositional possibilities, in favor of the compositional qualities of the 
face which multiplies the masks of presence thanks to the growing number 
of digital channels. 

As regards future lines of research, it would be necessary to deepen the 
understanding of the relationship between the expressive qualities of the 
face and images — in particular, to identify other kinds of masks produced 
by faces within images, in addition to the iconizing mask of portraits, the 
multiple experiential masks of selfies, the fictional mask of the filmic or 
theatrical character, and the socialized playful mask of influencers we have 
tried to describe. 


Notes 


1 This project has received funding from the European Union’s Horizon 2020 re- 
search and innovation program under the Marie Sklodowska-Curie grant agree- 
ment No 896835 — IMACTIS. 

2 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). The text is a re-elaboration of a talk presented during the international 
conference “Semiosis in Communication Culture, Communication and Social 
Change” held in Bucharest on May 25-29, 2021. The intervention was part of 
the panel “The Meaning of Artificial Faces” organized by Massimo Leone in the 
context of the ERC Consolidator project FACETS. 

3 We would like to refer to D’Armenio and Dondero (2020) for a semiotic analysis 
of casual, professional, and design photography practices on social networks. 

4 www.youtube.com/watch?v=qAW6CE7_bFM&t=1s 

5 We would like to refer to D’Armenio (2021 “L’économie”) for a more in-depth 
analysis of the experiential regression managed by algorithms on social networks 
and of the resulting economy of values. 

6 On visual semiotics, see Dondero (2020) and Basso Fossali and Dondero (2011). 

7 According to Eco (1975) a semiotic system is anything that can be used to lie. 

8 www.instagram.com/p/OovWwJHAG6f/ 

9 For an analysis of the new genres that are emerging on social networks, and their 
relation to the forms of identity narratives, we would like to refer to D’Armenio 
(2021 “La gestione”). 


5 Emerging faces 


The figure-ground relation 
from renaissance painting to 
deepfakes! 


Maria Giulia Dondero 


1. Introduction 


This chapter will address five theoretical attitudes with regard to portrai- 
ture. The images I will consider to exemplify these five theoretical posi- 
tions can be regarded as images embodying different theories on vision and 
identity. The statement according to which images embody theories (about 
looking, producing, observing, and so on) finds support in the semiotic 
theories of enunciation and of the metavisual proposed in several works by 
French School Semiotics (Fabbri 1998; Basso and Dondero 2011)? and that 
I present in my latest book The Language of Images. The Forms and the 
Forces (Dondero 2020b). To summarize, it may be said that every image 
proposes a specific position to the observer and thus a singular vision of 
the world and, notably, a manner of looking at this world. In this sense, 
every perspective within an image is a proposition of a way of conceiving 
of the world and, moreover, of conceiving of images as a way by which to 
envision the matters of perspective and of point of view — the position of 
seeing. In other words, the image is the concretization of a way of conceiv- 
ing of the image itself and an instrument by which to conceive of the world 
(including, in what concerns the specific matter of interest to this chapter, 
to conceive of identity). 

In this chapter, my fundamental argument will be that visual forms are 
made of forces, that is to say that forms are made of oriented markings. 
Every marking possesses a direction which generates flows of energy that 
may conflict with one another within the frame of the image. This concep- 
tion of the image as a container of energies that can be stored and reori- 
ented in the future has been proposed in Warburg’s theory of Pathosformeln 
and exemplified in his panels composing the Atlas Mnemosyne (2012): 
The form is indeed understood by Warburg as a deposit of forces that can 
be abated, reactivated, and transferred. The German theorist considered 
images to be “dynamograms”, that is, visual forms having registered a 
dynamic force that they are in turn presumed to transform and transmit to 
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other images. Another theorization of forces in images, especially in paint- 
ing, comes from art historian Henri Focillon (1934), who was the first to 
speak of a genealogy of forms and, in particular, of forms as resulting from 
the forces of the resistance of matter subjected to the technical work of 
designing and of composing. More generally, the frame can be considered 
as a device of enclosure that pushes the composition toward a stabilization 
of the forces and of the directions carried by pictorial markings. Forces are 
living and dynamic because they are what support the forms; their tension 
ensures the stability of the composition and sustains the manifestation of 
forms. 

These five theoretical attitudes that may be seen in various traditions of 
portraiture will be described through a panoramic overview that will look at 


. an example of Renaissance painting, 

. some portraits taken from the Baroque era, 

. the representation of heads in portraits by Francis Bacon, 

. Roland Barthes’ conception of the photographic portrait, 

. the contemporary facial images used to produce deepfake videos. 


ABR wWN eR 


As a conclusion, I will propose a mereological view of each of these five 
types of compositions, that is to say, how the relation between the total- 
ity and its parts is modeled and remodeled in these five different kinds of 
portraits with a special emphasis on the relation through which the figure 
and ground are respectively shaped. 


2. Renaissance portraiture: the plenitude of presence 


The first attitude I wish to describe is the one exemplified by Renaissance 
painting. I consider the Portrait of Baldassarre Castiglione by Raphael 
(1514-1515, Louvre, Paris) as a model of this attitude. The Renaissance 
portrait is the starting point of my reflection because it exemplifies very 
well the ideal-typical rules of classical portraiture. In fact, at first glance, 
classical portraits appear to be highly compact, not very articulated, and 
excessively immutable, especially if compared with the other great genres 
of the painterly tradition. It seems that portraits only generate a simple 
contrast, a binary figure-ground articulation which hinders the appre- 
hension of any narrative deployment of the image. This impression of an 
intemporal freezing can be explained through at least three rules? that are 
highlighted by the literature on this kind of portrait (Beyaert-Geslin 2017). 

The first rule concerns the relation between the figure and the ground 
and specifically the centrality of the face. A portrait is first and foremost 
the focalization of attention upon a subject. The ground, intended as a 
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worldly environment for the figure devoid of intensity, has to present itself 
in a very neutral manner so that the figure may emerge and be met directly 
by the observer. 

The second rule is the absence of action, that is, the pose. For the identity 
of the subject to become salient, his or her body must be immobile and not 
engaged in the performance of any action. The only relevant action would 
be gazing, the gaze being directed from the subject’s inner space towards 
the environing world, with the objective being to put forth something inti- 
mate and exclusively unique, in an offering to the observer. Any action 
other than the subject’s gazing would perturb this communication between 
the subject and the observer. As portrait theorist Jean-Marie Pontévia 
states, “in portraiture, the model is busy only with a single task: Looking 
like him or herself”.* The subject represented must therefore only display 
that part deemed to be the most “noble”, the part closest to the eyes and 
to the head because a view of the body would already carry the value of a 
proto-action or of a movement exceeding the simple act of gazing, thereby 
breaking away from the highest level of presence achievable.‘ 

The third rule, and in a sense the most important one for the Renais- 
sance period, is compactness. Through compactness, the face, that part of 
the body which is the most marked by past experiences, is able to condense 
a life story to be displayed before a viewer. In the painterly tradition, the 
compactness of the figure is considered to be a necessary feature of all 
portraits, and this also reflects a literary topos regarding portraiture: Each 
portrait is meant to condense the life of the subject represented. Perfection 
in portraiture is achieved when the subject succeeds in coalescing his or her 
self’s past experience and destiny in the here and the now of the act of being 
looked at, while ensuring the linkage of the moments of both the portrait’s 
production and observation. The face presented through a certain com- 
pactness or density on the plane of expression produces a “cohesive total- 
ity” against the void formed by the background. On the plane of content, 
the density and compactness of the figure must be capable of signifying the 
sum of the represented subject’s life experiences. 

The condensation of the figure obviously depends on the chromatic, 
luminous, and topological stability of the ground. It is therefore unsurpris- 
ing that the use of blurring has been excluded from portraits, as it would 
hinder the recognition and valuing of the subject, who by definition forms a 
well-determined and circumscribed totality. In portraiture, blurring would 
impede the subject’s stabilization and would suspend any exchange with 
the observer; what would result from blurring would instead be a floating 
identity. 

To summarize, we can say that the model of portrait that we inherit from 
the Renaissance is a portrait that has for task the presentation of a human 
being in his or her full completeness/plenitude by virtue of the fact that the 
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present time captured in the portrait genre condenses the subject’s past and 
the future experiences. These past and future experiences condensed in a 
present view are the exemplification of one’s destiny. To put it shortly, the 
portrait is an ideal and exemplary biography, an identitary narrative fixed 
in a perfect equilibrium between the figure and the ground, between the 
subject and his/her worldly environment. 


3. The baroque portrait: the struggle between presence 
and absence 


The next model I would like to briefly examine is that of the Baroque por- 
trait. A famous portrait by Caravaggio, Boy With a Basket of Fruit (1593, 
Galleria Borghese, Rome), can be seen as one of the examples of this paint- 
ing tradition. In this portrait by Caravaggio, the tension between the figure 
and ground is no longer irenic; there is a tension between the person rep- 
resented who has difficulty emerging as a clear and complete individuality 
and the background, where the shadows are not really realistic — they are 
geometrically unjustified. The background is unstable, almost magmatic, 
and it places the figure in an unsteady position. I propose to understand 
this portrait as one in which the manifestation of identity gives way to a 
struggle between presence and absence. That is to say that the portrait’s 
field of action is rather modulated by various gradients of intensity of pres- 
ence and absence by contrasting and competing forces (for instance, light 
and shadow). These competing forces prevent the observer from capturing 
the essence of the identity of the subject portrayed. 

The famous portrait Girl With a Pearl Earring by Vermeer (1665, the 
Mauritshuis, the Hague) displays as well something incomplete: the subject 
is evasive and presents herself in a sort of negation of exposure. The girl 
begins to retreat from the focal area of attention and to withdraw from the 
gaze of the observer; she is on the point of evading the shot (by turning 
away), and the observer is denied any power of vision. The background is 
completely dark and allows the painter’s vision to go no further. The ambi- 
ent darkness blinds the painter and the observer, dissolves the contours of 
the figure, and destroys its compactness. The treatment of the materials of 
the girl’s clothes gives a very ephemeral character to her appearance: it is 
as if she is about to evaporate, not only because she is turning away from 
us but also because the materials of her clothes look ghostly and seem to 
be on the verge of disappearing. A negation of presence is at work here. 
In another portrait by Vermeer (Study of a Young Woman, 1665-1674, 
Metropolitan Museum), we can notice the same functioning of the ground. 
We cannot really see where the lady’s hair ends and where the background 
begins, meaning that the figure is not stable but is rather fluctuating in 
an ether that does not freeze the face or, consequently, the identity. With 
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Renaissance and Baroque paintings, we have taken into account two dis- 
tinct models of portraiture: in Renaissance portraiture, the background 
enables the figure to emerge, and the identity is cast forth in its plenitude. 
In Baroque portraiture, on the contrary, the struggle between the figure 
and the ground on the plane of expression corresponds to an eroded and 
unsteady identity. 


4. The deformation portrait 


If the deformation of faces surely appeared in the Renaissance and Baroque 
periods as an eccentric representation method as we can see in this seminal 
portrait of Parmigianino (Self-portrait in a Convex Mirror, 1524, Kun- 
sthistorisches Museum, Wien), it was only in the twentieth century that 
the deformation of the face became a theorized and attested artistic pro- 
gram. I think of Francis Bacon who had his own theoretical ideas regarding 
portraiture and sensation in addition to the well-known interpretation of 
his painting by Gilles Deleuze. Deleuze is, if not the first, one of the best 
theorists of forces in painting. In his book Francis Bacon: The Logic of 
Sensation (2003), Deleuze defines the task of painting as the attempt to 
render visible forces that are not themselves visible. What is remarkable in 
his theory of the relation between forms and forces is that he distinguishes 
between transformation and deformation: 


These are two very different categories. The transformation of form 
can be abstract or dynamic. But deformation is always bodily, and it is 
static, it happens at one place; it subordinates movement to force... . 
When a force is exerted on a scrubbed part, . . . it turns this zone into 
a zone of indiscernibility that is common to several forms, irreducible 
to any of them; and the lines of force that it creates escape every form 
through their very clarity, through their deforming precision (we saw 
this in the becoming-animal of the Figures). 

(Deleuze 2003: 59, my emphasis) 


What I find remarkable in this idea is the fact that deformation is opposed 
to transformation and to an external dynamic: movement. The dynamic 
of forces is internal: internal to the face and to the head of these subjects 
who are stricken by deforming forces. As Deleuze states, deformation “is a 
movement ‘in-place’, a spasm, that reveals the action of invisible forces on 
the body” (Deleuze 2003: 41, my emphasis). Deleuze continues: 


[T]he extraordinary agitation of these heads is derived not from a move- 
ment that the series would supposedly reconstitute, but rather from the 
forces of pressure, dilation, contraction, flattening, and elongation that 
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are exerted on the immobile head. They are like the forces of the cosmos 

confronting an intergalactic traveler immobile in his capsule. It is as if 

invisible forces were striking the head from many different angles. 
(Deleuze 2003: 58, my emphasis) 


Deleuze states that a force is exerted on a body and specifically “on a point 
of the wave” (Deleuze 2003: 56). We are here at the antipodes of a well- 
deployed identity. The body and, in the cases of portraits, the face are con- 
ceived of as a wave and not at all as a stable entity. Bacon’s paintings show 
that to cover this wave, to make this wave present and sensible through 
visual means, it is necessary to abandon the figurative, the illustrative, and 
the narrative so as to attain what he calls the figure, that is, the Matter of 
Fact — the sensation. In a sense, we could say that illustration and narra- 
tive are like movement: something that occurs outside of the painting and 
that is not specific to painting; on the contrary, painting has the objective 
to capture and render the forces as “in-place movements”, spasms, and 
waves. To do so, it is necessary to drain the figure from every narrative: 
narrative is an obstacle to the capturing of deformation forces. The Matter 
of Fact that Bacon is looking for is a kind of separation and isolation of the 
figure from history, from the ordinary narrative of life, and from everyday 
worldly events. To reach the Matter of Fact means detaching the figure 
from every sort of narrative action: what characterized the Renaissance 
portrait (the absence of action) becomes in Bacon’s painting the objective 
of all types of paintings, including that of the triptych, where he obstructs 
every temptation the observer may have to narratively link the triptych’s 
panels and to build a story. But in contrast to Renaissance portraiture, 
reaching the Matter of Fact means achieving a portrait of sensation that is 
the opposite of a portrait that illustrates an identity. Portraits of sensation 
have to do with the existential void and torments of a subject: 


When he paints the screaming Pope, there is nothing that might cause 
horror, and the curtain in front of the Pope is not only a way of isolating 
him, of shielding him from view; it is rather the way in which the Pope 
himself sees nothing, and screams before the invisible. 

(Deleuze 2003: 38, my emphasis) 


As we can see in the Study After Velazquez’s Portrait of Pope Innocent X 
(1953, Des Moines Art Center, Des Moines), the objective that the Brit- 
ish painter has always pursued is to show the struggles between forces, 
between what is not immediately manifested, to paint the “violence of 
sensation more than the violence of the spectacle”, as he states through 
this beautiful formula: “The struggle with the shadow is the only real 
struggle”. The Matter of Fact, the sensation in a portrait, must overlook 
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the recognition of an individual, the biography of a person, and build 
what Deleuze calls a “diagram”. The diagram is generated by injecting 
chance into the painting through accidental marks.’ In fact, the painted 
face is the result of accidental marks that depend partly on the hand of 
the painter and partly on the chance which the gesture involves. This pro- 
cess is called a diagrammatical process: in order to destroy the figurative, 
that is, the fixed identity in the case of portraiture, the painterly gesture 
has to be controlled by chance. All these statements show that Bacon 
revolutionized the way of understanding portraiture by dissociating the 
portrait from the valorization of an identity considered as a complete and 
stable entity. 


5. Barthes’ Camera Lucida and his last seminars at the Collége 
de France 


Roland Barthes realized a comparable revolution in his publications on 
photography. In his book Camera Lucida, he opposes the identity of a 
person to his or her air. When Barthes speaks about the body and the face 
of a beloved person, as in the case of his mother, he states that the photo- 
graph must allow the discovery of that human being is in “its essence, ‘as 
into itself ...’... beyond simple resemblance, whether legal or hereditary” 
(Barthes 1999: 107). Barthes describes the air of a face as the opposite of 
resemblance, as something unanalyzable, as an evidence, and evidence is 
defined as “what does not want to be decomposed” (Barthes 1999: 108). 
Barthes describes the air as following in his Camera Lucida: 


The air (I use this word, lacking anything better, for the expression of 
truth) is a kind of intractable supplement of identity, what is given as an 
act of grace, stripped of any “importance”: the air expresses the subject, 
insofar as that subject assigns itself no importance. In this veracious 
photograph, the being I love, whom I have loved, is not separated from 
itself: at last it coincides. And, mysteriously, this coincidence is a kind 
of metamorphosis. 

(Barthes 1980: 109, my emphasis) 


And he expands on legal identity as a mask: 


All the photographs of my mother which I was looking through were a 
little like so many masks; at the last, suddenly the mask vanished: there 
remained a soul, ageless but not timeless, since this air was the person 
I used to see, consubstantial with her face, each day of her long life. 
(Barthes 1980: 110) 
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It’s interesting to note that this topic of the air captured by a photograph 
is something that makes one discover again what specifically identifies a 
beloved person, what is “consubstantial with [his or] her face, each day 
of [his or] her long life” (Barthes 1980: 110). Recalling the idea of the 
Renaissance portrait that had the role of depicting the destiny of a human 
being, we may ask what has changed since then. In Barthes’ conception, 
the air manifests itself as soon as the legal identity is overlooked. In con- 
trast, during the Renaissance period, it was the representation of the legal 
identity that had been privileged. For Barthes, the subject has to be in a 
way unconscious of him or herself: “the air expresses the subject, insofar 
as that subject assigns itself no importance” (Barthes 1980: 109). In this 
case, it’s the supplement of identity, something that could look like an 
ephemeral attitude but that becomes the center of the uniqueness identify- 
ing a beloved being. This supplement has no longer to do with the sum of 
past and future experiences because it comes from a subject’s not being 
conscious of him-/herself. This supplement of identity is a sort of moment 
of grace that is difficult to capture and difficult to reproduce: it’s the magic 
unique to a person.’ 

In another text written for his last seminars at the Collége de France, pub- 
lished posthumously in the book La Préparation du roman (2003), Barthes 
went even further and developed this idea of air in a more extreme way. 
A selection of photographic portraits, taken by Paul Nadar, which Barthes 
elected to show to his students, was found in the Proust archives; they 
represent aristocratic people who, according to Barthes, may have inspired 
some of the characters of the La Recherche du temps perdu (1913-1927). 
Despite the fact that the explanations of the photographed subjects are 
very precise about people who really lived, Barthes’ real goal was not to 
find physical and psychological resemblances between the subjects Marcel 
loved and the characters of the Recherche but, on the contrary, to uncover 
the atmosphere of a past world. 

In this text, Barthes proposed to extend the idea of air, which is defined 
here again as an “intractable supplement of identity”, a trace of the non- 
separation of the person from him or herself, something that cannot be 
quantified. But contrary to the theorization of the air in Camera Lucida, 
here, the air is no longer linked to a portrait of a person but to the portrait 
of a sort of a world, the world that Marcel Proust deeply loved. In this 
sense, the air is something that not only dissipates resemblance but also can 
be shared among a multitude of people. 

The air of a world, in this case something akin to a Proustian world, is 
something that is shared among people who have a common sensibility 
and who form a community of “Elective Affinities”. It means that the air 
of a world can be conceived of as the result of the de-personification of 
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individuals. Individuals don’t count anymore for their specificity, for what 
makes them different from one another, but for what they evoke to the 
observer when taken together. So an air is the result of a blending of many 
faces, where the individual face is no longer focused upon: in this process 
of blending, a face becomes fused with the other ones. We could speak of 
an ephemeral collective portrait, “ephemeral” because Barthes elected to 
successively show the slides of the faces without allowing time for his stu- 
dents to focus on any single one of them. The purpose of this seminar was 
not to analyze these represented faces but 


to intoxicate you [the students] with a world, as I am intoxicated with 
these photos, and as Proust was with their originals. Intoxicated with 
what? With the accumulation of these faces, of these glances, of these 
silhouettes, of these clothes; with a feeling of love towards some: one of 
nostalgia (they lived, and all are now dead). 

(Barthes 2003: 391-392, my translation)’ 


In this case, the relation between the figure and the ground is very peculiar: 
it is no longer the figure of an individual which is predominant: it is rather 
the ground of a collective group that becomes the figure of this beloved 
world - every individual figure needs to disappear, and the faces of each 
person must become interchangeable to allow the emergence of the ephem- 
eral aura of a collective portrait. 


6. The blending of faces in deepfake videos 


The de-personalization of the face described in Barthes’s last seminars 
leads us directly to another kind of de-personalization of the face, the one 
which takes place in deepfake videos, which consists in a process of face 
replacement — a face swap. We all know how deepfake videos work: Con- 
volutional neural network algorithms analyze a source face from differ- 
ent angles to learn what it looks like; then, the algorithms transpose these 
learned features onto a target face as if it were a mask. In a face swap,” a 
user feeds the encoded images of person A’s face into the decoder trained 
on person B. The decoder then reconstructs the face of person B with the 
expressions and orientation of face A and vice versa. The more convincing 
fake contents have been produced by extracting thousands of frames and 
are of course associated with “voice cloning”, which uses technologies for 
deepfake audio.'! What is important for achieving a very good replace- 
ment, as stated by the tutorials of the Faceswap app, is to use “as many 
different angles, expressions and lighting conditions as possible”. Because 
it is essential to “train the model to understand a face at all angles, with all 
expressions in all conditions, and swap it with another face at all angles, 


with all expressions in all conditions”. 
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The technical process of extraction comprises three steps: detection, 
alignment, and mask generation procedures. Detection is the process of 
finding faces within a frame while alignment consists in finding the land- 
marks within a face and in orienting the faces consistently. The landmarks 
of a face are the core elements, the key data points that indicate the facial 
position and expression in each frame. Ultimately, it is a reduction of facial 
information to a diagram which stores and computes every expression. 
To replace one identity, it’s necessary to work with the highest complexity 
of face gestures and movements. This complexity coincides with all the 
diversity that can be found within a single individual, and this diversity is 
rendered by a diagram made of landmarks." 

Finally, the procedure of mask generation creates a “mask” that identi- 
fies which parts of the final image are part of the face and which belong 
to the background or are obstructions. Here, the concept of background is 
treated together with the concept of obstruction, as noise alongside infor- 
mation. In a face swap, a mask is composed of the face’s saliencies against 
a background." The critical point, even in very contemporary representa- 
tions of the face, is always the relation between figure and ground, the 
fact that the background has to remain distinct and not disturb the face 
taken as a whole, even when this whole is not that of a single view upon 
an identity but the whole calculated on the basis of every occurrence of 
a face that will be swapped onto the face of someone else. This kind of 
mask is fundamental for our purpose because the background must be 
prevented from interfering with the detection of the figure and is treated 
as noise, as something that may prevent from correctly identifying the dia- 
gram of landmarks that define the essence of the face in deep learning 
processes. 


7. Conclusion: the whole to parts relations in portraits 


To conclude this chapter, I wish to recapitulate the different stages briefly 
presented through the perspective of the figure-ground relation. In the 
Renaissance period, portraits would represent the destiny of a person. And 
for this destiny to be understood as radiating from the figure, the figure 
and the ground were made to occupy different areas of the image. Every- 
thing is in its place, and the portrait shows the ontology of a human being 
in a perfect, irenic environment. It’s the portrait of the full presence of a 
deployed identity in a world that fits the figure. In the Baroque portrait, 
there is a negation of presence at work built upon a contrast in forces on 
the expression plane (i.e., light versus shadow) that means a contrast, on 
the content plane, between presence and absence, between the emergence 
and disappearance of presence. 

In the twentieth century, Bacon’s portraits were a continuation of 
the Baroque tradition and of the focus on identity transformation/ 
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deformation. The figure and ground are blended: the figure seems to be 
invaded by the ground, the ground being a force in place that acts upon 
the form of the face. A blending of the ground and the figure takes place, 
just as we have noted regarding the deepfake mask, but the difference 
is that in Bacon’s portraits, the ground melds with the figure, freeing it 
from the narrative and illustrative identity and conducing to what Bacon 
calls a portrait of sensation and a total negation of illustrative identity, 
whereas in deepfake videos, such invasion stems from the exclusive focus 
on saliencies so as to achieve a perfect diagram of the landmarks calcu- 
lated for the face. 

The portrayal of the air of a world, as Barthes had called for, also involves 
a sort of blending: a multitude of faces has the effect of producing a col- 
lective portrait of a whole, of a totality, and of a certain world. But such a 
world can only be suggested and not directly shown because Barthes sought 
to make the people represented lose their individual identity so as to form 
an atmosphere, something ephemeral, something that is not quantifiable, 
that is to say, something auratic. 

Conversely, in deepfake video production, the multitude of occurrences 
of a face is the means by which to determine, to calculate the landmarks 
of a person, the diagram of the face, making up a silhouette of points that 
build a mathematical totality. This totality is schematic and not a full total- 
ity because the noise formed by the ground now encroaches upon the figure 
as it incorporates the non-fundamental part of the visible face, that is to 
say, the part that is left to the indeterminate and to the Other. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 

2 Close to semiotics, other scholars have worked on the image as a concretiza- 

tion of a theory. Among them, we may notably think of Damisch (1987) or the 

photographer Denis Roche. 

For more details on these rules, see Dondero (2020, The Language). 

Our translation. See Pontévia (2000: 16). 

This may differ in some extra-European cultures of the face. For instance, Indi- 

ans ask their photographers for complete body portraits, and, as Pinney (1997) 

wrote, they threaten to pay the photographer half the price if only half of the 

body is taken in the picture. 

6 Ihave already expressed myself regarding this matter (Dondero 2021, Compo- 
sition). This paper focuses on the meaning of the blurred portrait in a dialectic 
between artistic and scientific photography and between the portrait of an in- 
dividual and the portrait of a human type, especially in the case of the experi- 
ments of Francis Galton (composite photography). 
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On this matter, see the vol 9, issue 1 of the journal Metodo. International Stud- 
ies of Phenomenology and Philosophy on “Diagrammatic Gestures” (Dondero 
and La Mantia, Eds. 2021) and especially the papers by Batt and Dondero 
which focus on Deleuze’s theory of forces in painting. 

This state of grace has been defined as being the characteristic of beauty in Don- 
dero (2022, Models), where the beautiful woman’s face coincides with some 
kinds of movements and unique ephemeral gestures. The paper concludes as- 
serting that this ephemeral character of beauty in photographed faces is more 
similar to the description of the instable equilibrium made of air and water in 
Chinese landscape painting than to the Western tradition of the figure/ground 
division in portraiture. 

On Barthes’s idea of photography as it is conceived of in his last seminars at 
College de France, see Dondero (2006, Barthes). For a very detailed descrip- 
tion of the scientific attitude and teaching approach of Barthes during his last 
seminars, see Amigo Pino (2018, 2022). 

Cfr Momina Masood, Mariam Nawaz, Khalid Mahmood Malik, Ali Javed, 
Aun Irtaza, Creation of a Deepfake using an auto-encoder and decoder. The 
same encoder-decoder pair is used to learn the latent features of the faces during 
training, while during generation decoders are swapped, such that latent face 
A is subjected to decoder B to generate face A with the features of face B. Ex- 
tracted from Momina Masood, Mariam Nawaz, Khalid Mahmood Malik, Ali 
Javed, Aun Irtaza, “Deepfakes generation and detection: state-of-the-art, open 
challenges, countermeasures, and way forward”, draft: www.researchgate.net/ 
publication/349703826_Deepfakes_generation_and_detection_state-of-the- 
art_open_challenges_countermeasures_and_way_forward, 2021. 

For instance, Lyrebird AI has the capacity to form sentences just by hearing a 
few spoken words, thus creating a digital voice fake. 

See https://forum.faceswap.dev/viewtopic.php?f=5 &t=27 

This conception of the diagram is slightly different from that of Deleuze, men- 
tioned in the previous pages. For a discussion on the similarities and differences 
between the conception of the diagram in Deleuze and Goodman, see Dondero 
(2023, The Experimental). It is rather more inspired by Goodman’s (1976) the- 
ory of the diagram, according to which the diagram may be seen as a third term 
between dense (autographic) and notational (allographic) symbolic systems: 
dense systems are well illustrated by pictorial arts constituted by non-repeat- 
able entities (such as each unique face), whereas notational systems, which are 
articulated and composed of elements that are fixed value, disjointed, and recom- 
binable, concern arts such as music and architecture, in which the score or the 
architectural project resorts to an articulated language (mathematics or algorith- 
mic work), namely a notational system providing unambiguous instructions for 
each new execution. Goodman does not however assimilate the diagram among 
the allographic arts. He rather includes the diagrammatic among dense and au- 
tographic systems, placing it at the extremity of a gradient of which the opposite 
pole corresponds to the pictorial. Both the pictorial and the diagrammatic are 
thus made to form part of the same gradient of dense systems where they oc- 
cupy the two opposite extremities. Here, the diagram is the mediator between 
the uniqueness of every face and the mathematical calculation of proportions 
within a face: in this case, the geometrization of the facial landmarks constitutes 
a diagram because it generates at the same time a rarefaction of the specificity of 
a face and a densification of the simple calculations of its proportions. 
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14 Cfr Github Face Swap. The model performs its full potential when the input 
images are preprocessed with face alignment methods. https://github.com/ 
shaoanlu/faceswap-GAN/blob/master/README.md. 

15 The students attending the seminar were also supposed to lose their institu- 
tional identity so as to become the “intoxicated” people desired and loved by 
Proust. Barthes had the objective of making them into a blended collectivity of 
individuals (see Barthes 2003; Dondero 2006, Barthes). 


6 ‘Timely faces! 


Antonio Dante Santangelo 


1. Symbolic “artificial” faces 


Every now and then, when watching the media, we come across faces that 
significantly embody some cultural models that are crucial to understand- 
ing the meaning of our times. They are “artificial” in the sense that even if 
they belong to actual people, they are the result of representational work. 
They are symbolic because they signify some abstract concepts that are 
central to our culture in a certain historical period. It is in this sense, for 
example, that Roland Barthes once remarked that the faces of Greta Garbo 
and of Audrey Hepburn, in the various movies circulating in the fifties, 
admirably represented that particular moment when cinema was about to 
extract an existential beauty from an essential one (Barthes 1957). 

The faces I want to talk about are of this kind. They are those of the 
young protagonists of La vie d’Adéle — Chapitres 1 & 2 (Kechiche 2013) 
and Jeune Femme (Serraille 2017), two movies awarded at the Cannes Film 
Festival, respectively, with the Palme D’Or and the Caméra d’Or prizes. 
Their representation led several Italian film critics to argue that “exag- 
gerating Flaubert, Adèle c'est nous [Adèle is us]”,” since the girl embodies 
“the basic problem of the loneliness of the contemporary individual, of 
the misery of the proposals for liberation and for full and profound self- 
affirmation that this era offers” > Other critics, also forming part of the Ital- 
ian scene, considered Paula, the hero of Serraille’s film, as “a matrix upon 
which a face selected at random from the batch of contemporary European 
youth could be affixed”.* For them, she is “one of the most authentic female 
characters of recent years”, one who “symbolizes the shouts of the new 
generations wanting to claim their place in the world”.® My hypothesis is 
that if Adéle and Paula look so meaningful, in the Italian cultural context, 
it is because they both represent a typical strategy that many persons fol- 
low to find a sense to their life in a society where social mobility — or, more 
generally, conceiving one’s own existential project, believing that one day 
it will be achieved — has become almost impossible. 
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In fact, if we look at their stories, the protagonists of La vie d’Adeéle — 
Chapitres 1 & 2 and Jeune Femme are some kind of vagabonds. From the 
very beginning of the films in which they appear, they do not know who 
they actually are or who they want to become. So, like many people of 
their age, they seek their own path, experimenting freely with the many 
opportunities offered by Paris and Lille, the towns where they live. Between 
heterosexual and homosexual love affairs, jobs, and homes of varying pres- 
tige, the two girls, both of whom come from the suburbs and from the 
petite middle class, understand that they are different from the people who 
populate their places of origin. Hence, they try to settle into the upper class 
because their significant others belong to it. Nevertheless, they realize that 
the world is made of rigidly separated niches and that, even if these niches 
seem to be open to everyone, in many of them, they cannot find a place for 
themselves. This ends up causing pain to Adéle and Paula, but it does not 
lead them to make any political demands. It is simply that, at the end of 
their stories, they find themselves alone, and they have to continue on their 
way without yet having understood where they will go. 

Since they do not have clear existential projects, the two girls look for 
the meaning of their life by trial and error. They seem to believe that it will 
eventually manifest itself as they accumulate the most varied experiences. 
This is the reason why the sensorial organs of their faces — their mouths 
and skin, in particular, but even Paula’s heterochromatic eyes — are very 
central to their representation, so much as to become the main subjects 
of the posters for the movies. They are a metaphor for the difficulty faced 
by the two protagonists of La vie d’Adéle — Chapitres 1 & 2 and of Jeune 
Femme in following one single ideally predefined trajectory. Rather than 
distantly looking towards the endpoint of an existential path upon which 
to center their life, they have to grope for the meaning of the many situ- 
ations in which they find themselves immersed. In practice, paraphrasing 
Barthes again, they symbolically represent that particular moment when 
cinema was about to differentiate the notions of the meaning of life either 
as something towards which to aim or as something that one does not 
know where it can be found, but that may suddenly become manifest as 
one wanders through the world. In fact, a synonym of the word “mean- 
ing” is “sense”: sense as a direction that can no longer be pursued and that 
transforms itself into a sensation, the last refuge for the existential quest of 
the new generations who can only dare — or maybe gamble — to find it this 
way, especially if they do not belong to the upper classes. 

To demonstrate that this interpretation of the two movies and of the 
iconic faces of their protagonists is correct, I will compare them to the 
contents of another film about a young woman who is very representative 
of our times: Chiara Ferragni, the famous influencer. I will then analyze 
Chiara Ferragni — Unposted (Amoruso 2019), the documentary film that 


Timely faces 89 


tells the story of her life, showing how she started off as an outsider to the 
world of fashion and how she achieved glory and fame counting only on 
her talent. Once again, in this movie, the representation of the protago- 
nist’s face is central in particular as regards her beautiful blue eyes that 
so clearly see what she wants and that direct the choices she has to make 
in her private and professional life. So much so that her eyes — or maybe, 
more significantly, if compared to the two-colored eyes of Paula in Jeune 
Femme, one of them — become the logo of her fashion company. Since she 
was young, Chiara Ferragni knew that she liked fashion, and eventually, 
she became a star in the field. 

What interests me here is that many commentators in Italy have criti- 
cized the message conveyed by Chiara Ferragni — Unposted, defining it as 
something of a fairy tale that may be misleading for many young spectators 
who might believe that their life can also be such. But in many reviews, the 
authors pointed out that the famous influencer is the daughter of a wealthy 
family and of a mother who already worked in the fashion industry. More- 
over, they stressed that great marketing strategists have always been help- 
ing her. Actually, these critics, even without referring to La vie d’Adeéle — 
Chapitres 1 & 2 or to Jeune Femme, seem to share the same worldview as 
the new generations depicted in these films, to the point where they believe 
that real life is more similar to the fictional stories of which the films tell. 
They radically contrast the world view portrayed therein with the one pur- 
sued by persons such as Chiara Ferragni, who they judge to be even more 
“fake”.’ Hence, I will compare the cultural model of the self-made woman, 
which the story told in Chiara Ferragni — Unposted clearly embodies, to 
that of the aforementioned vagabonds, showing how central their opposi- 
tion is for understanding the meaning of our experience of life nowadays, 
at least in the Italian context. 


2. Some preliminary theoretical considerations 


Before delving into the analysis of the faces and films I have written about, 
I feel it necessary to offer some indications as to the method that I will use. 
It is a method that derives from the structural semiotic tradition, and it is 
based on the idea that when we need to interpret the world, we recur to 
signs and narratives that are in our mind and that we use to classify what 
we are faced with (Ferraro 2012). Such signs and narratives make sense 
because they are encoded into structures we all know and share. These 
structures put them into some logical relations, most of the time in opposi- 
tion to one another (Saussure 1916; Hjelmslev 1943). For example, if we 
see the image of a tiger in an adventure movie and want to understand its 
meaning, in our mind, we will oppose it to the images of herbivorous or 
domestic animals. Then, we will oppose the stories about the relationships 
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between humans and wild predators to those between humans and harm- 
less animals. Similarly, if we see a face in another movie, we will recognize 
the signs used to create it but also the narrative structure of the story of its 
owner. Doing so, we will recognize the face as being similar to some other 
faces and different from some other ones, and this operation will enable us 
to give it a meaning. 

Starting from these convictions, Lévi-Strauss (1979) once demon- 
strated, in his studies on Salish swaihwé masks, that although they may 
look very different from the Kwakiutl dzonokwa ones, they are actually 
similar because they have the same meaning for the populations who pro- 
duced them. At the same time, although they have many traits in common 
with the Kwakiutl xwexwé masks, they are actually different because the 
stories of the characters they represent talk about the same concepts, but 
they have opposite narrative structures. What I want to do with my anal- 
ysis is analogous. However, instead of studying exotic cultures, I want to 
show how Italian culture works to make the faces of the protagonists of 
La vie d’Adéle — Chapitres 1 & 2 and Jeune Femme look similar, even if 
the two actresses who embody them are physically different. At the same 
time, even if, like Chiara Ferragni, Adéle, and Paula are two young and 
beautiful women, I want to demonstrate that their faces have the same 
meaning as one another because the signs used to represent them and the 
narrative structures of their stories are in similar opposition to the ones 
used in Chiara Ferragni — Unposted, which are typical of another kind 
of stories. 

Nevertheless, behind my analysis, there is another theoretical convic- 
tion. It is connected to the explanation of the fact that, as I have men- 
tioned earlier, the critics of the three movies have affirmed that the two 
fictional characters which are Adéle and Paula are “real”, in the sense that 
they actually resemble the young people forming the new generation while 
Chiara Ferragni, who claims to tell her own true story, is not believed. In 
fact, many think that she has somehow “fictionalized” her chronicle, mak- 
ing it look like something of a fairy tale. Actually, as I will show better in 
the following paragraphs, fairy tales can tell the true meaning of life as 
envisioned by a certain culture. For example, Propp (1946) demonstrated 
that the narrative structure of Russian magical fairy tales reminds of the 
initiation rites seen in many other parts of the world, and anthropolo- 
gists know that initiation rites, like myths and other narratives, reproduce 
the vision of reality upheld by the populations they belong to (Van Gen- 
nep 1909). This is due to the presence, behind every factual or fictional 
discourse about our way of being in the world, of some similar cultural 
models that, like signs and narratives, are tools that we use to classify 
our experience of life by opposition, so as to give it meaning. Such cul- 
tural models are actually made of signs and narratives. In fact, elsewhere, 
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I define them as recurring discursive configurations with specific narrative 
structures that connect some signs so as to give them a certain connotative 
meaning (Santangelo 2012). We recur to them every time we have to talk 
about a particular topic, and they enable us to take a position towards it. 
Hence, the problem with Chiara Ferragni— Unposted is not that it conveys 
a fairy tale but that such a fairy tale embodies a cultural model which is 
no longer believable for portraying the meaning of the life of the young 
people of today because it is too elitist. It is more realistic, when talking 
about this topic, to construct discourses such as the ones we can find in 
La vie d’Adéle — Chapitres 1 & 2 and Jeune Femme because the class of 
persons whose existential experience has a similar meaning is wider. In the 
following pages, I will try to show how the cultural models from which the 
three movies derive oppose one another, why they are so central to under- 
standing our times, and how they determine the representation of the faces 
of their protagonists. 


3. Two similar stories 


As I wrote before, a cultural model is a semiotic tool that enables us to 
interpret some seemingly different discourses in the same way, focusing on 
their analogies more than on their differences. We give such discourses the 
same meaning because we recognize that they are based on similar narra- 
tive configurations of the same signs. Therefore, to become aware of the 
existence of cultural models and to describe them, we have to find that 
different texts on the same topics function in the same way. This is exactly 
what happens if we compare La vie d’Adéle - Chapitres 1 & 2 and Jeune 
Femme.’ 

As I have anticipated, both of these works are about young women 
trying to define their identity, to find their place in the world. These 
girls come from petit bourgeois families on the outskirts of Lille and 
Paris, and they both experience leaving the environment in which they 
were born. They both enter into relationships with artists belonging to 
the upper middle class: Paula falls in love with Joachim, an older estab- 
lished photographer, while Adèle falls in love with Emma, a painter who 
is also a little bit more mature and who is studying at the Academy of 
Fine Arts. Both, however, experience being violently ejected from the 
lives of their respective lovers: Jeune Femme begins just like this, with 
the door of Joachim’s flat closed in Paula’s face and with her screaming 
and banging her head against this barrier to get it reopened, injuring her 
forehead. La vie d’Adeéle — Chapitres 1 & 2, on the other hand, gradu- 
ally reaches this situation after the protagonist, having moved in with 
Emma, cheats on her with a colleague and, having been discovered, is 
sent away. 
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The similarities between the two films continue. Paula begins to wander 
from one social context to another, looking for a place to settle down. She 
finds two jobs: one as a babysitter for the daughter of a wealthy woman, 
who also gives her a place to sleep in the attic of her beautiful house in the 
center of Paris, and another as a saleswoman in a shopping center, where 
she meets Ousmane, a black security guard, separated and with a son, with 
whom she starts a fleeting relationship. However, Paula also seems to fall 
in love with Yuki, a girl she met by chance in the metro, with whom she 
explores the world of nightclubs. She tries to get hosted by Joachim’s rich 
friends. She even agrees to see her former lover again, when he backtracks 
and goes looking for her. Finally, she also meets her mother, with whom 
she has a stormy relationship and who would like her to return home, but 
she decisively flees. 

Adéle, for her part, is no less of a wanderer. In the first “chapter” of 
her existence — the film portrays her as a teenager and then as an adult — 
she accepts the courtship of Thomas, a schoolmate, who is deemed highly 
desirable by her friends. The two make love, but the spark does not ignite, 
at least for Adéle. She realizes that she feels attracted to women and, in 
particular, to Emma, the nonconformist blue-haired painter, who she finds 
to embody the ideals of beauty and freedom that she reads about in her lit- 
erature classes or in her philosophy books. However, after letting herself be 
swayed by passion and moving in with her partner in a social context very 
different from the one from which she comes, she does not feel understood 
or valued, except as the object of Emma’s own artistic expression. The two 
women, animated by irreconcilable existential horizons, drift apart until 
the aforementioned betrayal, perpetrated by Adèle with a man whom, for a 
moment, she feels closer to. In fact, he belongs to the world she has chosen 
for herself by inclination, that of teaching children. Only, having been dis- 
covered and then driven away, Adèle feels she wants to get back with her 
former lover at all costs — a hope that, unfortunately, proves to be in vain, 
as Emma makes a new life for herself with another woman who, like her, is 
well connected within the world of art. 

Both Adèle and Paula, therefore, “wander” between one life experience 
and another, accepting and transgressing rules and clichés, in search of 
their own identity, which they do not find. In this regard, both films end 
with the protagonists having yet another strong and significant experience, 
one that is painful but not resolved. For Paula, there is her choice to abort 
alone in a hospital, renouncing the child she conceived with Ousmane. For 
Adèle, there is her participation in Emma’s first triumphant vernissage, in 
which she finds out that she is now but a past memory for the artist, perpet- 
uated through the paintings hanging on the walls. Hence, they both end up 
walking down the street thoughtfully, alone, passing through anonymous 
places, where they feel they do not belong. 
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4. A similar narrative structure 


The narrative structure of the two movies is shown in Figure 6.1. To under- 
stand it, it is necessary to know that among the narrative structures to 
which we recur, particularly relevant for the study of faces, we have those 
of Alpha class stories (Ferraro 2019: 96-98). Such stories focus on how 
to determine the identity of their main characters (and we all have a sense 
of how closely faces are tied to identity), depending on the relationship 
between an individual value system, also called the perspective instance, 
and a collective value system, called the sending instance (Ferraro 2012: 
172-174). The former gives rise to all the parts of this kind of story in 
which we talk about the meaning that things and events take on in the eyes 
of their protagonist, based on his or her innermost personal convictions. 
The latter, on the other hand, is responsible for the meaning that those 
same elements of the narrated story acquire for the other characters who 
constitute the social reality in which the protagonist operates, who have 
their own way of seeing. Between these two instances, various types of 
relationships can exist, and such relationships play a fundamental role in 
determining the identity of the characters who embody them. For example, 
such characters can be those who fight for their individual value systems as 
opposed to a collective value system they consider to be unjust. Conversely, 
they can be the individuals whose personal value systems coincide with 
those of others. In other words, they can be subjects with an individual 
value system that differs from the collective one, subjects who decide to 
not oppose to it, and so on. It is clear that even the overall meaning of the 
events of their stories depends on the relationship between the sending 
instance and the perspective one. 

In La vie d’Adéle - Chapitres 1 & 2 and Jeune Femme, the perspective 
instance represented by the two protagonists and their desire for free- 
dom push them to search for themselves regardless of social expecta- 
tions. However, this desire clashes with the sending instance represented 
by the social rules that inform the various niches of society in which 
the young women try to settle down. The characters who populate them 
would like to define the identities of Adéle and Paula, telling the girls 
who they should be, but this does not make sense to them. For example, 
when Adèle affirms that she dreams of becoming an elementary school 
teacher, Emma urges her to rather write novels because in the upper class 
to which she belongs, people admire artists who express themselves more 
so than teachers who repeat the words of others, taking care of the educa- 
tion of children. The fact that she paints Adéle in her works also allows 
her to define the identity of her lover and to share it across her own 
social context, regardless of what the latter thinks of herself and wishes 
to show publicly. Paula, for her part, lives through a similar experience, 
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as she too is “defined” by Joachim’s successful photos and as her partner 
expels her from his life, once again because he judges her unsuitable for 
him. In fact, he has a prestigious job and clear existential goals while she 
is unemployed at the beginning of the film, having just returned from a 
trip abroad in search of herself. However, when Joachim tells her to go 
to work, it does not make sense to Paula because the girl has no idea of 
what she would like to do. 

Unlike their lovers, Adéle and Paula do not know who they are and 
would simply like to understand it through their own self-experience. 
Only, they realize that this is complicated if even love is governed by the 
values of the social contexts to which the people they meet belong, and 
as soon as they stray from them to try something else, they are brutally 
thrown out. They also experience what it means to be part of the world 
of their parents, work colleagues, schoolmates, and friends, who all do 
the same thing: they attempt to judge and define who the girls should be. 
Unfortunately, they both fail to adhere to any of the models they come 
across because within the boundaries imposed upon them, they struggle 
to recognize their own identity. Hence, they end up “wandering” between 
one sending instance and another (this is the meaning of the different 
arrows in Figure 6.1). However, they remain cut off (this is the meaning 
of the horizontal line in Figure 6.1) without ever finding a home for good. 

This narrative structure is perfectly recognized by the Italian film critics, 
who consider it decisive in its capacity to tell what life is like for young 
people today. Of the protagonist of Jeune Femme, for example, someone 
writes that 


undecided with respect to her means, her affections, her ambitions, Paula 
is the mirror of a generation that no longer identifies with the bour- 
geois ideals and aspirations of the second half of the 20th century, . . . 
yet is too weak to propose an alternative model.’ 


Sending instance Sending instance Sending instance 
1 2 N 


Wes ee 


Perspective instance 


Figure 6.1 Antonio Santangelo. 2022. The narrative structure of Adéle’s and 
Paula’s stories. Copyright: Upon a kind concession by Antonio Dante 
Santangelo. 
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Her wandering between one sending instance and another, in search of her 
own identity, may be traced back to the aesthetics of flânerie, so often used 
by the directors of the French nouvelle vague. But one commentator rightly 
points out that 


compared to the films of the Young Turks, Serraille’s is the child of a 
more written and less free script, as well as of a worldview in which 
the protagonist’s bewilderment and her losing herself in the nonsense of 
everyday life is not a declaration of anarchy with regard to the rules, but 
rather a way of re-entering them.!” 


Paula, in fact, does not wander because she is non-conformist, but only 
because she does not understand, a priori, what should be the sending 
instance that establishes the values of that part of society in which she 
wishes to live. Thus, she tries different paths, but they never take her 
where she really wants to go: “throughout the film, she never delivers 
her character to a recognizable or calculatingly lovable model, remaining 
a constantly changing, metamorphic, elusive character, in transition at 
every level”.!' 

The same themes are identified and considered significant by critics of 
La vie d’Adéle — Chapitres 1 & 2, confirming the fact that the story told 
by director Abdellatif Kechiche, being the fruit of the same cultural model, 
appears superimposable to the one narrated in Jeune Femme. Many, in 
fact, emphasize Adéle’s great openness and her desire to experience life 
without prejudice. However, they highlight the castrating and in many 
ways unjust nature of those social mechanisms that cause suffering to 
young people today: those who, like Adèle herself, seek their own way 
freely, outside of the socio-cultural context from which they come, and 
who meet with existential failure. They find themselves “wandering” 
between very different portions of the world, in which it is not possible to 
settle down definitively. In this regard, a commentator points out that the 
girl does not follow “a traditional evolutionary trajectory, which involves 
a series of changes in worldview until the final attainment of a certain 
maturity and emotional equilibrium”.” If this were the case, at the end of 
the film, she would find her own identity. Instead, as I have hinted, Adéle 
ends up experiencing “the basic problem of the loneliness of the contem- 
porary individual, of the misery of the proposals for liberation and for full 
and profound self-affirmation that this era offers”,'’ as is always the case 
in Kechiche’s cinematography. In fact, it all seems to be centered upon 
“social classes as the regulators of an order that is difficult to negotiate; . . . 
and on slices of life . . . never understood as simple sections of everyday 
life, but rather as emblematic segments of formative paths that do not 


produce evolutions”.'* 
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5. Faces of vagabonds 


The narrative structure of La vie d’Adéle — Chapitres 1 & 2 and Jeune 
Femme is represented by analogy by a set of signs that are used several 
times throughout the two films, also giving shape to the protagonists’ 
faces. For example, since they do not identify a priori a sending instance 
pointing them towards a precise existential direction, Adéle and Paula 
need to find a way a posteriori. Therefore, as I have anticipated, the rep- 
resentation of the two girls is characterized by the emphasis placed on the 
details of their sense organs. In fact, they move by trial and error, testing, 
with their bodies even before engaging in rational planning, the conse- 
quences of what it means to live in the various social contexts with which 
they are confronted. 

All the reviewers of the two movies underline the symbolic importance 
of this emphasis on the bodily sensations of Adéle and Paula. Some point 
out that Adéle “reads the world only through her own body, made into an 
instrument of exploration of phenomenal experience” because for her, 
“existence comes before essence; that is, experience is not made compre- 
hensible by a prior definition of the self, but on the contrary, subjectivity 
is determined by the multiplicity of experiences themselves”.'® At the same 
time, Paula is also a character who “finally sees and perceives the world 
with the eyes and body of a woman, a real woman, who is not afraid to 
be what she feels she is”.'” In particular, since the face is perhaps the main 
sensory terminal of the human body, all critics of La vie d’Adéle — Chapi- 
tres 1 & 2 and Jeune Femme recognize that the portrayal of the protago- 
nists’ faces is crucial. Adéle “is constantly filmed with close-ups or details 
that enhance her overflowing physicality, her humoral fluids, from tears to 


mucus, the sense of a warm and deep instinctuality”." 


She is voracious, as suggested by Kechiche’s constant return to her lips 
filmed while she sleeps or eats: lips stretched out, greedy, almost distinct 
from the rest of her body, as if living a life of their own and dragging the 
protagonist forward against her will.” 


Once again, even Paula 


struggles to stay within the frame, due to the latter’s inability to contain 
her overflowing physicality. Before sinking the camera inside the girl’s 
story . . . director Léonor Serraille keeps to the surface of events and to 
the wounded and impatient body of actress Laetitia Dosch.”° 


In particular, she highlights her heterochromatic eyes, “ ‘bipolar eyes’, mir- 
roring an equally bipolar and restless soul”.7! 
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As can also be seen in the posters for the two films, the sensory organs of 
the faces of Adèle and of Paula, particularly the mouth of the former and 
the eyes of the latter, in addition to the excoriated skin on her forehead, 
are fundamental in symbolically communicating the meaning of the two 
girls’ stories. Adéle and Paula, in fact, enter into relation with a material 
reality that functions according to the values of the sending instances that 
give a meaning to the people, places, and even the food they find therein. 
Adèle, at the beginning of Kechiche’s work, is shown enjoying the “simple” 
pasta cooked by her father with details of her teeth chewing voraciously 
and her chin getting dirty from too much eagerness. However, when she 
visits Emma’s more upper-class parents, she is shown tasting oysters, a 
notoriously more “refined” dish, which she reluctantly savors. When she 
makes love to the young painter, she voluptuously probes the latter’s body 
mainly with her mouth, and when she betrays her and is expelled from her 
life, she tastes, again with her mouth, the bitter taste of her own tears and 
the mucus that drips down her nose. It becomes clear then that there is an 
opposition in the two films between, on the one hand, the lips and skin of 
the protagonists and, on the other hand, their eyes. The former represent 
the sense, also in its sensorial meaning, which is found a posteriori, link- 
ing these two components of Adéle’s and Paula’s faces to their perspective 
instance, an instance with which, paradoxically, is associated a kind of lack 
of perspective, an inability to look afar, beyond the limited horizon of the 
here and now. The eyes, on the other hand, refer to the various sending 
instances found in the world in which the girls live. They are gazes upon 
the latter, institutionalized and defined a priori, capable of indicating a 
direction — like those of Emma and Joachim, who not surprisingly make 
art by observing and defining reality from their own point of view, which is 
already “successful” or is soon to become so because they know it is shared 
with all those with whom they keep company. However, Adéle and Paula, 
more or less metaphorically, have changing eyes, which do not lead them 
to see things in the same way as those whose irises are of a single color. 
They are different: as “vagabonds”, they do not look at the world from 
a single perspective but try to identify with the point of view of different 
sending instances. Such an attempt is certainly full of strongly flavored 
sensory stimuli, but it is also characterized by the concrete risk of “banging 
one’s head” and getting hurt. 

As is thus evident, the signs that make the sensory characteristics of the 
faces and bodies of the protagonists of the two films pertinent stand for 
something else. They take on a second and symbolic meaning, clearly linked 
to the narrative structure of the story they help to tell and, in particular, 
to the relationship that, within the story itself, is established between the 
perspective instance and the sending one. This is also the case of the signs 
that emphasize the movement, made by both Adèle and Paula, between one 
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social context and another, as well as being the case of the movement of 
the strictly handheld camera that shoots them. These signs are also given 
a central role in communicating the deeper meaning of the discourse pur- 
sued by the directors of the two films. In this regard, I have already had 
the opportunity to write about the extent to which critics of Jeune Femme 
appreciate Serraille’s expertise in the use of a handheld camera. But the 
same thing is emphasized in reviews of La vie d’Adéle — Chapitres 1 & 2, 
pointing out that “the camera, rarely placed far from the characters’ faces, 
is a seismograph with an icastic and immediate sensitivity; every change of 
mood is detected in real time”.”? The expertise recognized in Kechiche lies 
in his ability to always know how to capture the manifestation of mean- 
ing, at the exact instant when it is consciously detected by the protagonist 
experiencing it with her own body and the organs of her face. However, 
to achieve this, he too must make himself into a “vagabond”: he cannot 
direct the camera towards a specific point, knowing a priori that he will be 
filming something significant there. Rather, he must always be on the move. 

Closely related to these signs are those that communicate a feeling of 
incompleteness, of both Adéle’s and Paula’s lives and of the images that 
represent them. Indeed, the handheld camera, focusing more on the details 
of the two girls’ faces than on the whole scene, makes it difficult to con- 
struct complete, balanced, stable, and harmonious shots. Moreover, since 
completeness, balance, stability, and harmony are not part of the existence 
of the protagonists of the two films, these concepts are not sought for on 
a visual level in order to tell their stories. If Adéle and Paula succeeded in 
integrating their perspective instance with one of the sending instances of 
the world in which they live, if their identity allowed itself to be “framed” 
within the logic and values of one of these sending instances, then the way 
of filming the two young women and, more generally, of narrating them 
would be different. But Serraille, when interviewed about the underlying 
message of her film, asks: “What does it mean to be a young woman? 
We are often asked to fit a certain pattern, an identity, a definition. The 
expression “Young woman’ should be a free, deliberately undefined one”.”° 
And indeed, La vie d’Adéle — Chapitres 1 & 2 is so titled precisely because 
Kechiche does not intend to tell the whole story of his heroine’s life, prefer- 
ring to leave it unfinished, in the midst of her wanderings. This is some- 
what similar to the unfinished work La vie de Marianne (Marivaux, 1731- 
1742), one of the books Adéle studies at school, drawing inspiration from 
it for her existential quest. 

Finally, in these two cinematographic works, the signs that communicate 
separation are fundamental. The authors of the reviews of La vie d’Adéle — 
Chapitres 1 & 2 emphasize the importance of the director’s insistence on 
using very narrow shots when capturing his film’s protagonist, keeping 
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her figure separate from the context in which she lives. This is underlined, 
for example, by those who appreciate Kechiche’s expertise “in depicting 
the details of faces and bodies, in lingering on the very near close-ups, 
and then widening the shots to other things, where strictly necessary, only 
when others become an attraction for Adéle’s gaze”.** Because “there is 
nothing beyond what the young woman knows and experiences”*: “more 
light, more details enter the field of vision, but also blur things in the back- 
ground”.’° At the same time, the commentators of Jeune Femme point out 
how Paula “is almost always filmed alone”,’” emphasizing her difficulty in 
integrating with her surroundings. Even in these cases, the images depicting 
the faces and bodies of the protagonists of the two films, so clearly sepa- 
rated from those of the other characters who populate their world as well 
as from the spaces the latter occupy, symbolically represent by analogy the 
separation between the perspective instance and the sending one around 


which their stories are structured. 


6. The story of a self-made woman 


The cultural model that serves as a matrix for the stories of vagabonds and 
the representation of their faces contrasts sharply with that of the stories of 
self-made men and self-made women. To demonstrate this, one only has to 
look at how Elisa Amoruso’s Chiara Ferragni — Unposted is constructed. 
Here, it is shown that Chiara, ever since she was a child, has always had 
only one desire: to follow in the footsteps of her mother, a well-known pro- 
fessional in the fashion world, in order to establish herself in this context. 
Thus, the perspective instance, represented by the individual values and 
convictions of the protagonist of this film, fits perfectly with the sending 
instance with which she is confronted. The latter constitutes, for her, an 
existential horizon that is valid a priori, one that is capable of filling each 
of her life choices with meaning. Her identity depends, then, on her ability 
to break through the barrier I mentioned, describing the narrative structure 
of vagabond stories: the “young woman” recounted by Amoruso, so dif- 
ferent from those in the works of Kechiche and Serraille, must succeed in 
gaining a prominent role in the society of professionals who share her same 
passions. Here, she gains citizenship thanks to her talent and dedication, 
driven primarily by her fans on social networks, the first to recognize her 
qualities. Thanks to their support, Chiara progresses through her cursus 
honorum, from her first steps as a fashion blogger to her achievement of 
fame, with the numerous attestations of esteem received from the great- 
est stylists, managers of the most famous labels, models, and specialized 
journalists, but also with the foundation of her own brand of clothing and 
accessories, her marriage to the well-known singer and showman Fedez, 
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the birth of her son Leone, and the moving of her whole family to Los 
Angeles, where she keeps company with the biggest stars of the interna- 
tional jet set. 

In Figure 6.2, one can visualize the narrative structure of the story told 
in Chiara Ferragni — Unposted and some of the many images within the 
film that represent it by analogy. As is natural, given that it is a mat- 
ter of recognizing the identity of a young “outsider” within the world 
of fashion, a prominent role is played by her face, “framed” within the 
most representative spaces of this context, be they the covers of glossy 
magazines, boutiques, catwalks, or the headquarters of the most impor- 
tant companies. 

However, I will return later to the representation of Chiara’s face. For 
the time being, it is interesting to emphasize that the story of her life fuses 
the vicissitudes of the well-known influencer with those of the heroes 
of the Russian magical fairy tales analyzed by Propp (1928). In fact, it is 
no coincidence that Ferraro, in his book on theories of narration (Ferraro 
2015: 75-85), defines them as self-made men. As Propp himself argued 
in a later work (Propp 1946), they are reminiscent of the figures of those 
young men who, wishing to show that they are worthy of reproducing the 
system of values established by the sending instance of the social context 
in which they aspire to fit. They undergo a sort of initiation rite, facing 
hard individual trials and thus pursuing some form of collective recogni- 
tion, especially by those who are already in a prominent position within 
that society. 

As I have shown elsewhere (Santangelo 2013), Hollywood textbooks 
have taken up this narrative matrix (see, for example, Vogler 1992) in 
order to show American film scriptwriters the way to construct success 
stories, endowed with a strong symbolic meaning for viewers all around 
the world. After all, watching films about characters such as those depicted 
in fairy tales, who start out poor and become rich, who begin their journey 
outside of a certain social system and who end up, owing to their own 
merit, at the center of it, can be edifying for many. All of this, in fact, 
embodies the American dream. But ultimately, it represents the desire for 
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4 


Perspective instance 


Figure 6.2 Antonio Santangelo. 2022. The narrative structure of Chiara Ferragni — 
Unposted. Copyright: Upon a kind concession by Antonio Dante 
Santangelo. 
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affirmation held by any of those who believe they can find their true selves 
by adhering to the values shared by those who belong to that niche of soci- 
ety in which they aspire to live. 

In this sense, then, Chiara Ferragni — Unposted fits into an ancient nar- 
rative tradition. Indeed, reading the Italian reviews of this film, this matrix 
and the cultural model of which it is the bearer are recognized by all. Yet 
the choice of resorting to it to tell the story of a real person, with whom so 
many young people identify, has been strongly criticized with arguments 
such as the following: 


If we talk about affirmation and personal fulfilment . . . the film certainly 
clarifies one point, namely that the idea that anyone, with commitment 
and determination, can realise their dreams is still a very pervasive con- 
cept, but above all, a narrative that guarantees a multi-million-dollar 
box office. . . . The real knot where the film proves artificial is . . . the 
message behind this empire. You can make it on your own. You don’t 
need anybody. Believe in your dreams and anything can happen... . 
And to say that the documentary shows Chiara as a child, filmed and 
photographed by her beautiful mother, who came from the fashion 
world. We will see a rich and happy family. . . . Above all, we will see a 
host of ubiquitous, friendly and diligent collaborators, even ready to tie 
her shoes. . . . Making it on their own? It is certainly noble to fight for 
the idea that women can be completely autonomous and independent, 
but perhaps those who have only lived in such a privileged dimension 
should not speculate on such dangerous illusions . . . after having seen 
the film, the dream machine increasingly seems like a deadly factory, 
ready to incessantly generate frustration and sadness.” 


Indeed, there are also those who are thrilled to finally see on the big screen 
the story of a girl who succeeds in realizing her dreams, as indicated by 
those who praise Chiara Ferragni — Unposted. They deem it 


almost a miracle, precious in entering a reality far away from us, to 
show the unique value of an entrepreneur, blogger, model and designer 
who has revolutionised the world of fashion, overcoming the barriers to 
entry of a closed and snobbish sector, uncomfortable because it shows 
the face and impact of a true influencer.”? 


However, both of these types of interpretations — of which the proponents 
of the latter openly polemise with those of the former, writing that “nega- 
tive reviews by critics are . . . quite negatively biased due to a lack of 
understanding of the phenomenon”*? — can only be fully understood if 
one compares them to the vagabond stories I wrote about in the previous 
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paragraphs. In fact, for those who believe that the cultural model and 
matrix of stories such as La vie d’Adéle — Chapitres 1 & 2 and Jeune 
Femme accurately describe the condition of young people today, the refer- 
ence to the American dream achieved by the famous Italian fashion blog- 
ger cannot but appear “fake”. It looks like the result of fairy tale imagery, 
good only for international box-office success and for generating false 
myths, coming from a cinematography that is currently incapable of tell- 
ing how the world is really going. Therefore, even those who believe that 
Chiara Ferragni — Unposted represents a breath of genuine novelty know 
that they are confronted with the opposite cultural model, which is the one 
shared by most Italian critics. 


7. The face of a self-made woman 


Since the narrative matrix of Chiara Ferragni — Unposted contrasts with 
that of La vie d’Adéle — Chapitres 1 & 2 and Jeune Femme, the same hap- 
pens with the signs that, within Amoruso’s film, represent by analogy the 
structure of the story being told. These signs, as in the works of Kechiche 
and Serraille, in turn, are also used to construct the image of Chiara Fer- 
ragni’s face. 

Firstly, the body and face of the famous influencer are never separated 
from the background of the social context in which she lives because it is 
important to show her integration within the latter. Moreover, the cam- 
era that shoots her is rarely mobile but is fixed on a tripod, also signifying 
the stability of the condition of the film’s protagonist, who has arrived 
where she wants to be. Chiara allows the spectators to observe her with 
ease and to pry into her world while explaining how she got there. Only 
in a few moments, in the archive images shot by her mother with an ama- 
teur camera when Chiara was a child, do we find the handheld camera 
aesthetics of La vie d’Adéle — Chapitres 1 & 2 and Jeune Femme. Sig- 
nificantly, this is the case in the scenes that tell of when the protagonist 
of Amoruso’s film was not yet famous and when those watching her, 
a bit like Kechiche and Serraille with Adéle and Paula, were trying to 
understand what her identity would be in the future. However, precisely 
because Chiara has now found herself, the director who tells her story 
chooses to talk about her completeness as a person and the completeness 
of her existence. She does so by assembling harmonious and balanced 
shots, perfectly in line with the balance and harmony of the hero of her 
film, magically capable of holding everything together, from success to 
family, from the pride of her Italian-ness to the internationality of her 
fame, from friendship towards her employees to the ability to direct them 
like a great manager. 
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Above all, however, instead of speaking, as in La vie d’Adéle — Chapitres 
1 &@ 2 and Jeune Femme, of bodily sensations, of mouths, and of epider- 
mises seeking meaning in the here and now, Amoruso focuses on Chiara’s 
capacity for vision. This is represented, as I have mentioned, by her beauti- 
ful blue eyes that know how to look far ahead and by the logo of her label, 
which, uncoincidentally, is represented by an eye resembling her own. It 
is these eyes, set within a serene and satisfied face framed with composure 
and precision, that symbolically represent the narrative structure of the 
cultural model from which Chiara Ferragni — Unposted derives. In this 
film too, there are in fact signs that focus on the bodily sensations and the 
skin of the protagonist. At the beginning of the narration of her story, she 
wants to get a tattoo to forever remember the happy moment she is going 
through. At the end, she sits by the sea, enjoying the pleasant breeze of the 
Californian beach, close to the place where she has made her home. But it 
is precisely in this last sequence, in which Chiara scans the horizon, imagin- 
ing what the future will be like, that we understand the difference between 
Adéle’s and Paula’s life story. We find ourselves, in fact, at the end of a path 
of personal fulfillment of which Chiara herself enjoys the results, which 
remain imprinted in her body and soul, making her confident in what is to 
come. In the works of Kechiche and Serraille, we are rather in the middle 
of the journey. We are in the midst of the wanderings of two young women 
who are more or less the same age as the protagonist of Amoruso’s film 
and who could legitimately aspire to be self-made women but who are still 
in search of themselves and of their place in the world. What they lack, 
precisely, is a vision, capable of holding together their perspective instance 
and the sending instance that governs the social contexts in which they 
attempt to integrate. 


8. The eye model and the tongue model 


The analysis I have conducted so far shows that the opposition between the 
vicissitudes and faces of vagabonds and those of self-made women is par- 
ticularly significant in the Italian culture represented by the reviews I have 
quoted, as if there were a code for reading reality that opposes the narra- 
tive models I have described in Figures 6.1 and 6.2. However, the same 
code opposes vision to taste, stability to movement, and completeness to 
incompleteness in addition to opposing integration to separation from the 
social context within which one lives. 

With this in mind, I find it interesting to cite the research of Gobbi and 
Morace, two authoritative observers of the changes in cultural models, 
which they study in order to intercept new trends in the field of consump- 
tion. These authors argue that we live in an era in which the logic that has 
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characterized the history of our way of consuming over the last twenty 
years, based on 


the eye and fashion, on image and on pure visual sensibility, has entered 
a crisis. In this model, personal status reigned supreme . . . expressed 
through a landscape of objects and products that remained outside of 
our tactile, bodily, kinesthetic experience, without ever coming into 
contact with our epidermis, our skin, our hands. 

(Gobbi and Morace 2007: 84-85) 


In the past, we lived in an economy of the eye and of sight, symbolized by 
the passion for clothes and clothing accessories indeed but more generally 
by status symbols. In such an economy, reigned the tale of someone who 
had arrived where he wanted to be and who, because of this, wanted to be 
admired, positively “sanctioned” because he wore the signs of his success. 
The contemporary era, on the other hand, is characterized by an economy 
of the hands and tongue, symbolically represented by the passion for food. 
It is linked to sensory experience in its making: 


[T]hought descends from personal experience . . . the real becomes a 
single, immense, permanent ice cream . . . language also refers directly 
to other experiences . . . first of all, to the experience of conversation, 
of speech, of dialogue . . . and then, even more profoundly, to an erotic 
experience with reality, in which what we like is worth licking and 
savouring. 


(Gobbi and Morace, ibid.) 


As can be seen, many of these concepts are common to the analyses 
I have conducted so far. It seems as if the consumption of a film such as 
Chiara Ferragni — Unposted would suit those who prefer the model of the 
eye, whereas the appreciation of works such as La vie d’Adéle — Chapitres 
1 & 2 and Jeune Femme would stem froma way of seeing the world that is 
more akin to that derived from the model of the tongue. Of course, the crit- 
ical undertones towards our society, which can be found behind the stories 
of Adéle and Paula, as told by Kechiche and Serraille, or behind the forms 
of criticism directed towards the discourse carried out by Amoruso, in her 
narration of Chiara Ferragni’s life, are not seen in Gobbi and Morace’s 
reflections. After all, consumer studies do not have this function. Even if, 
by watching the movies and reviews I have analyzed, one could argue that 
paradoxically, the self-made man or woman model nowadays may look 
more meaningful to upper-class people, who have the means to make it 
become true, whereas lower-class people have to switch to the vagabond’s 
cultural model. In any case, it is true that something leads many observers 
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of contemporary culture to say that our logic of reading reality is changing, 
and this reverberates widely, including in our way of thinking and symboli- 
cally interpreting faces. 


9. Old and new models of meaning construction 


Considerations such as those I have just written about make us reflect on 
the fact that the Strega prize for the best Italian novel of 2017 — one of the 
most prestigious literary prizes in Italy - went to Paolo Cognetti’s Le otto 
montagne’! (2016). The story is about a young man who searches in the 
mountains, amidst nature and old traditions, for the meaning of his being 
in the world. However, he does not find it and realizes that his destiny is to 
be always on the move, wandering like a vagabond among the metaphori- 
cal eight mountains that recall a Tibetan myth. Cognetti himself, in 2018, 
published Senza mai arrivare in cima. Viaggio in Himalaya,” declaring that 
he was inspired by Peter Matthiessen’s The Snow Leopard (1978). Accord- 
ing to Cognetti, in describing Matthiessen’s pilgrimage to Tibet, this book 
had captured the true essence of life. In this regard, the Italian writer states: 


Peter had used a very precise word for his journey. Gnaskor, which 
means to wander. .. . A pilgrimage is in every culture a path of puri- 
fication, but in wandering, in walking in circles, there is no point of 
arrival, which is fundamental in the pilgrimages we understand. Jeru- 
salem, Rome, Mecca: without a destination, how do you know when 
you have achieved purity? I found a connection between this need for 
holy cities at the end of the path and the mountaineering obsession 
with mountain peaks: since I was a child, I had heard the word sum- 
mit used as a metaphor for paradise, and the word ascent in a spiritual 
sense . . . on the other hand, the most important Tibetan pilgrimage is 
to go around Mount Kailash, which is sacred to that culture. Kora in 
Tibetan, circumambulazione*® in Italian: Christians plant crosses on top 
of mountains, Buddhists draw circles at their feet. I found violence in 
the first gesture, kindness in the second; a desire for conquest versus one 
of understanding. 

(Cognetti 2018: 20-21) 


As we can see, albeit with the due differences that need to be grasped 
among all these discourses on who we are and on the value of our experi- 
ences in today’s world, the opposition I wrote of in the first paragraph 
of this article has become the matrix of many successful contemporary 
works. On the one hand, there are those who speak of a sense of life found 
a priori, within a system such as that of our society, in function of a send- 
ing instance upon which the latter is centered — a sending instance that 
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is capable of indicating a path towards a visible destination such as, for 
example, a distant peak, reachable by those who decide to align themselves 
with their own perspective instance. On the other hand, there is a sense of 
life recognized a posteriori after espousing and departing from different 
sending instances, some central and shared by many in our own society, 
others more peripheral and minoritarian, others still completely external, 
coming from elsewhere. 

The question remains as to why the opposition between the cultural 
models I have written about seems so significant in justifying the interest 
of the works that speak of it. During my discussion, several explanations 
could be discerned. There is the more political one, for example, about 
a Western world that would have us believe that it is structured to pro- 
vide everyone with the opportunity to thrive within a system of values 
that would claim to be both liberalist and libertarian but which is rig- 
idly divided into classes separated from one another. This would cause a 
disconnect with respect to the narrative of self-made men and self-made 
women I have defined, justifying the emergence of the idea that our socie- 
ties are populated by vagabonds condemned to search for meaning that 
they struggle to find. In this regard, there is a very interesting sequence in 
La vie d’Adéle — Chapitres 1 G 2. We see the protagonist who, after hav- 
ing lunch at home with her petit bourgeois parents while watching TV, 
finds herself at school listening to a lecture on the aforementioned La vie 
de Marianne, a work that belongs to a more “elevated” culture and that 
teaches her to desire the very things she will later see embodied in Emma. 
This will allow her to enter into a relationship with the young artist, but 
it will not be enough to keep the two girls together. The attempt at social 
integration made by the school system, therefore, will fail due to a series of 
inequalities external to it. 

Then there is the socio-economic explanation. According to this perspec- 
tive, people are aware that they can find very different products and ser- 
vices on the market, all of which are of value, quite accessible, and capable 
of making the consumer imagine what it means to lead a certain “lifestyle”, 
even if it is far removed from their own. They would therefore no longer 
accept to “enclose themselves” within the existential horizons of only one 
of them, however desirable, but would rather be inclined to experiment 
with all of them in search of the one that best suits them. Rivers of ink 
have been spilled over the “incoherent” post-modern consumers (Fabris 
2003), who could very well be symbolically represented by Adéle’s bisex- 
ual orientation or by Paula’s heterochromatic eyes. Certainly, Gobbi and 
Morace’s reasoning, which I wrote about in the previous paragraph, goes 
in the direction of reading the meaning of our life choices in this key, which 
is much less critical of our societies. 
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Lastly, there is the anthropological explanation provided by Cognetti. 
According to him, the encounter with other cultures, at the end of a not- 
too-happy experience within the confines of our own, would have allowed 
us to leave the Christian perspective (of Weberian memory?). Such a way 
of seeing would be aimed at by laboriously following the indications of a 
sending instance shared by all, capable of suggesting in which direction to 
go and to assign a meaning to our life choices. However, today we find it 
meaningful to embrace another way of seeing, of oriental inspiration, one 
which is more inclined to “wandering” in a search for the self as a form 
of epiphany. In this regard, the Italian writer, as anticipated, commenting 
upon Matthiessen’s work, which he feels to be close to his own sensibility, 
says that “the 1970s, with their failed promises, had disappointed him, 
or perhaps he himself was the source of his own disappointment: he was 
entering middle age and realized he had done little” (Cognetti 2018: 17). In 
short, after having tried several paths and still not having found his place in 
the world, he attempts yet another, believing or hoping that this will be the 
right one. However, just like Matthiessen and Cognetti, he does not do so 
in order to leave his society forever but rather to do something meaningful 
to him and to a niche of people within society itself, such as writing a good 
book that someone can appreciate. Doing so, he still wishes to be part of 
the collective, as a representative of one of the many sending instances that 
are recognized within it. Of course, there are no guarantees of success, but 
wandering still makes sense. 
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The Eight Mountains, in English. 

Without Ever Reaching the Summit. Journey Into the Himalayas, in English. 
In English, “circumambulation”, as in “going around”, “walking in circles”. 


7  Featureless faces 
A film aesthetics! 


Bruno Surace 


1. Introduction: can a meme kill? 


Can a meme kill? It would seem so, according to a semiotically significant 
news story. Marcel Danesi, commenting on the nineteen stab wounds suf- 
fered by a twelve-year-old at the hands of two of her peers in Waukesha 
(Wisconsin) in 2014,” affirms that “the world of the matrix is more real and 
perhaps more meaningful to people today than the real world” (2019: 64). 
The girl survived by a miracle, and her friends stated they had performed 
the act as a sacrifice for “Slender Man”. It was therefore in all respects an 
attempted human sacrifice, made to please (or because forced by) an entity 
that ontologically does not exist, and that is part of an “online mythology” 
capable of inducing a form of so-called screen paranoia (ibidem). Slender 
Man does not exist in reality: we can identify its demiurge in Erik Knudsen, 
and we even have access to the reconstruction of some sources that prob- 
ably inspired this character’s creation: 


At that time, Knudsen used the username “Victor Surge”, a fact that 
seems to refer to the sources of inspiration for the visual design of the 
Slender Man. Described as a tall man, dressed in a black suit, and who 
has a “faceless face”, whose identity it is therefore impossible to recog- 
nize, the characteristics mentioned still point to a visual similarity with 
the character The Question (as it was called in Brazil). Typically a hero 
with no special powers, the Question was a masked vigilante who used 
his intelligence and combat skills to fight criminals. His secret identity 
was protected by a mask that made his face look flat. His real name 
was Victor Sage, a detail that probably influenced Eric Knudsen’s choice 
of pseudonym Victor Surge, as well as the lack of facial appearance in 
Slender Man. 

(Bastos Dias 2019: 261)? 
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However, Slender Man evidently exists as a cultural fact and has very pro- 
found consequences on the ontological reality. We are not interested here 
in thinking about the psychic disorders of the two stabbers — limiting our- 
selves to noting the relevance of this paranoid act’s having been carried out 
by not one but two people (a so-called folie à deux) of very tender age —* 
but rather in how what is in effect a fruit of fiction and human intelligence 
was able to become the motive (or at least the justification) for such a bru- 
tal act.° It is not the only related case: 


In May 2015, the New York Times reported that in the previous six 
months there had been nine suicides and over 100 suicide attempts 
made by youths living on the Pine Ridge Indian Reservation. As 
reporter Julie Bosman (2015) explains, “Several officials with knowl- 
edge of the cases said that at least one of the youths who committed 
suicide was influenced by Slender Man, a tall, faceless creature who 
appears in storytelling websites, often as a figure who stalks and kills 
victims.” 

(Blank and McNeill 2018: 14) 


Our thesis is that not only the narrative substrate but also the somatic 
specificity of this entity affects its success and can furnish some relevant 


Figure 7.1 A cosplay of the Slender Man. Credits: Terry Robinson; www.flickr. 
com/photos/suburbanadventure/8494619702; licensed CC BY-SA 2.0. 
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data in terms of a semiotic anthropology of postmodernity. Slender Man 
is a meme of a certain kind. It is undoubtedly the best-known exponent of 
that para-literary form born within the Internet known as creepypasta.” 
Creepypastas (portmanteau of “creepy” plus “copypasta”, an Internet 
slang expression)? are horror stories, usually written anonymously and 
disseminated online, starting from a dedicated “wiki” (https://creepy- 
pasta.fandom.com/wiki/Creepypasta_Wiki). The objects of these tales of 
terror are highly varied, but it is no coincidence that some recurring iso- 
topies of this fringe of “digital folklore” (see Sanchez 2018) appear in 
them: naturally the theme of the paranormal declined in many ways but 
also a certain protagonist dimension of the media (especially digital) as a 
vehicle of evil, the presence of adolescent or infant victims/executioners (a 
direct reflection of empirical/model authors and readers),’? and a certain 
tendency towards seriality (usually achieved through the recurrence of 
some characters, such as Slender Man) as a trace of that narrative eco- 
system that forms and conforms to the tastes of the new generations with 
increasing vigor.'° All this is situated on common ground, namely that 
of writing almost always devoid of strong literary connotations — there 
is a preponderant interest in content and plot rather than in expression 
or style — and of a cultural sediment which is a direct consequence of the 
so-called urban/metropolitan legends of the pre-digital world. In the light 
of these considerations, a further underlying characteristic of this genre is 
explained, and that is, a certain disregard towards the historical sources 
of what are always presented as natural narratives or, in other words," 
true stories: 


The Slender Man Mythos unquestionably functions as a virtual world in 
Saler’s sense: it is a fictional universe which countless individuals from 
around the world have chosen to both inhabit and build upon through 
creative contributions in the form of videos, games, and written narra- 
tives. The overwhelming majority of these contributors and participants 
also exhibit what Saler would term ironic belief: they know that Slender 
Man is a fiction, but winkingly create media that pretend otherwise. But 


as we have seen, not all responses to fictional media are ironic. 
(Tolbert 2015: 50)? 


All this is true although technically they are often false uchronic narratives 
as demonstrated by all the creepypastas that are based on Nazism, claim- 
ing to build on a historically solid background (since it is surreptitiously 
assimilated by readers who thus believe they have a true idea of the histori- 
cal event) when they are rather the result of the unconscious processing of 
a mythical imaginary already explored in Nazisploitation or similar veins 
several decades ago." 
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In this fictional context, Slender Man has assumed, from the year of 
its birth, 2009, to the present day, a mythological status, to the point of 
having been relocated transmedially numerous times, becoming a charac- 
ter in video games, cartoons, and even live-action films. It is a malicious 


Figure 7.2 An artwork depicting Slender Man. Credits: LuxAmber; https://commons. 
wikimedia.org/wiki/File:Touxuii_senosex.jpg; Licensed CC BY-SA 4.0. 


Featureless faces 113 


and mysterious entity, clearly humanoid in shape. An oblong, lanky “indi- 
vidual”, dressed in a suit and tie, with a variable number of elegant ten- 
tacles protruding from its back, but above all with a face/no-face. Slender 
Man’s head is white, and where the features should be, there is nothing. 
The creature is endowed with various paranormal powers and usually tar- 
gets youngsters or children, kidnapping them or driving them to insanity or 
self-injury. Its fame is such as to have generated a complex symbolism and 
an articulated narrative system of roles linked to its figure. 

The Slender Man schematically described earlier as “tall, dark, and 
loathsome” cannot but call to mind several previous references: it is 
very similar to a contemporary version of the Boogeyman, for example, 
but also conveys multiple traits of a horror imaginary that consoli- 
dated itself throughout the twentieth century in literature and fantasy 
cinema. The somatic datum, however, is what is most pertinent here. 
In fact, the character’s features in no way resemble those of a classic 
monster, except for its tentacles, an iconic remnant of a Lovecraftian 
background. There are no suckers or burrs. Slender Man’s tentacles 
are elegant symmetrical filaments, befitting a certain underlying grace 
inherent in the character. What makes the entity frightening is rather 
the physical disproportion of its stature, slightly taller than an ordinary 
human, and above all the absent face, which can only activate a doubly 
disturbing mechanism: 


1. Where there should be a certain system of human apparatuses, percep- 
tually there is nothing, which places the being in a categorical dyscrasia; 

2. Where there is nothing, those elements that serve as an “agency detec- 
tion” portal are missing, and so we fall back into a specific Uncanny 
Valley. 


The fact that it is clothed in a suit and tie lends itself to numerous ideo- 
logical readings but undoubtedly heightens that strange and paradoxical 
human/nonhuman condition that finds its confirmation and acme in the 
face. If we think of Slender Man’s visage in terms of a Greimassian articula- 
tion, then it is not the opposite of a face but its contradiction.'¢ It is a non- 
face, just as a zombie is not a dead man but a non-dead being. The fact that 
this specific non-face is thus placed in the axis of the subcontraries, that is, 
in that semiotically rather problematic gray area, only further motivates its 
esoteric charm. It is not in fact a representation of the indefinable face, like 
that — we will get to it — of Marcel Schwob sans-gueule (1891) but of the 
indefinite face. Slender Man’s face is a consubstantial non-face, nor can it 
be otherwise. The faces of the sams-gueule are, at least in their aspectuality, 
faces that are no longer faces. The two are quite different, although their 
disturbing effects may appear similar. 
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2. Slender Man — the movie 


The Slender Man phenomenon becomes a feature film of the same name 
with a major production in 2018, directed by Sylvain White. The result is 
not exciting, according to the critics who generally treat it as an unsuccess- 
ful experiment which wastes a potentially interesting character in rather 
trivial ways. In fact, the film does not stand out either from a formal point 
of view or in terms of the plot, re-proposing the tired pattern of the teen 
movie with the theme of evocation. Nonetheless, beyond these value judg- 
ments, in the text we find a whole series of elements that corroborate the 
theses put forward earlier about the characterizations of the Slender Man 
myth.!” 

There is certainly the theme of “media contagion”. The narrative prem- 
ise is that Slender Man persecutes those who are daring or reckless enough 
to summon it, an evocation that occurs “virally” — here is the underlying 
memetic humus —'* through a video circulating on the Internet which con- 
tains the secret instructions for this operation. Nothing new, if one thinks 
of cinematographic cornerstones such as Ringu/The Ring (Nakata 1998; 
Verbinski 2002), Poltergeist (Hooper 1982), Videodrome (Cronenberg 
1983), or lesser-known films such as Pontypool (McDonald 2009), Cell 
(Williams 2016), and so on. There the “traditional” media were the vehi- 
cle while here everything passes through the Internet and the discourses it 
produces: 


[A]s a consciously-constructed sign, Slender Man .. . reflects important 
semiotic processes at work among members of the various internet com- 
munities in which his [sic] legend has appeared. The most significant 
of these processes, and perhaps the area that has seen the most fruitful 
overlap of folkloristic and literary semiotic investigation, is ostension. 
(Tolbert 2013) 


Indeed, the Slender Man in the film is not a mystery. Students talk about 
it in high school, and the idea that it can be evoked with a specific ritual, 
transmitted through a sort of “electronic gospel” (Mellor et al. 2016), cir- 
culates as an initiatory myth. In this case, too, an eternal return is reified if 
one thinks of how many times in past decades the topos of the séance, held 
by adolescents in search of a thrill, has constituted the basis for demonic 
narratives of various kinds. Here, however, there is — in essence at least — 
the exaltation of a meta-discursive component. The film was born from 
an Internet phenomenon, to which it refers as a culturally existing fabric 
in the diegetic premises and even reproduces creepypasta wiki-style sites, 
showing the young protagonists, before and after the evocation, feverishly 
in search of videos to unravel the mystery of the creature that haunts them, 
in what at times therefore presents itself as a detective story. 
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In this spiritism 2.0 the medium therefore not only confirms its role as 
a vehicle of evil but is also promoted as an investigative tool. The role of 
the face is clearly paramount, as has already occurred in other films, such 
as Unfriended (Gabriadze 2014), entirely based on a Skype video call in 
which the ghost of a cyber-bullied girl punishes her peers by making them 
suffer via webcam, or Host (Savage 2020), with a similar plot expressed 
in a Zoom call.” In Slender Man, however, the face assumes various roles. 
There is the face filmed in the video-selfies, which become evidence of close 
encounters with Slender Man in the woods; the face in close-up, seen dur- 
ing sleep troubled by the nightmares induced by the creature (echoing Wes 
Craven’s Freddy Krueger); the non-face of the creature itself, here whitish 
and with veins that also make it a sort of mummy; or the faces belonging to 
the hallucinatory states of the protagonists who see strange figures with a 
void instead of a face or perceive people’s faces as deformed (in this vortex 
of madness sometimes the faces turn black or are penetrated by tentacles), 
as happens in other teenage horror movies such as Truth or Dare (Wadlow 
2018) from the same year, where the deformation of the face is a disturb- 
ing element and prelude to nefarious events. Already in Final Destination 
(Wong 2000), however, the initiator of a successful saga, those predestined 
for death have seen their faces dim in their photographs, and in Smile (Finn 
2022) an eerie smile is the prelude to a sordid, deadly curse. 

The somatic relevance of the character is further testified by a cer- 
tain iconic obsession that accompanies it. Slender Man in all respects is 
glimpsed, dreamed, and perceived through various visual indexes (shad- 
ows, traces of its appearance in loved ones, and so on). This obviously 
applies online, and in fact the web is full of videos and photographs more 
or less clearly created ad hoc in which the character can be discerned, and 
of course a similar semiotic protocol reemerges in the film. The characters 
draw it, look for evidence online, and so on. The creature’s specific design 
also makes it particularly inclined to materialize in various pareidolias, 
just as its reference “habitat” constitutes the mimetic fabric within which 
it can most visually manifest itself. Among the branches and brambles that 
become misshapen dark tangles in the night, the creature seems to be every- 
where because of its black dress and oblong limbs. The film will eventually 
merge environment and character when the latter reaps its last victim by 
literally incorporating her into a tree trunk. 

The non-face is the culmination, the ultimate place of making contact. 
Here, therefore, emerges a further specificity of Slender Man. One should 
not look it in the face, otherwise madness and, sooner or later, death will 
follow. So much so that the second evocation in the film takes place right 
in the woods, where the creature is thought to reside, but with the girls 
blindfolded. This veiling strategy is in effect a modification of the face 
through the affixing of a drape that at first sight appears useless, as the eyes 
already have their own biological ability to eclipse, namely the closing of 
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the eyelids. However, the tension between fascination and terror is evident, 
that ancestral curiosity towards the unknown, which requires bandaging 
as a form of artificial or prosthetic eyelid. Furthermore, this intensifies on 
the face a strategy of not seeing which is also, phenomenologically, a primi- 
tive form of not being seen. Just as children hide under their bedsheets to 
protect themselves from the monster under the bed, or a cat tries to conceal 
itself behind a post, the underlying mechanism is a sort of suspension of 
perceptual disbelief: 


Being seen (being witnessed) . . . in non-judgmental supportive somatic 
explorations is an antidote for depression and builds confidence. Being 
seen is ultimately about seeing as well. As phenomenology has taught 
[...| we cannot separate ourselves from the world. We are implicated 
from the start, as part of the otherness that we perceive to be separate. 
We are not alone; separateness is an illusion. . .. We might feel alone, 
however, as a matter of experience. Surely many people feel isolated, 
and empty sometimes, even those who are for the most part happy. Feel- 
ings of isolation and separation arise phenomenologically — as ways in 
which the world appears to us and is sometimes experienced. 

(Fraleigh 2019: 91-92)° 


Hiding one’s face, or part of it, blindfolding oneself, masking oneself, 
means not seeing but also not being seen, incorporating in the facial sur- 
face — a place of unification of the human senses and antechamber of the 
most vital of the organs, namely the brain — the wholeness of the body. And 
yet the bandaged face is also an immediately mutilated face, which protects 
itself from its own scopophilia but which is also, paradoxically, exposed. 
Slender Man thus engages in an atrocious form of blackmail: if one wants 
to dialogue with it, one has to deprive oneself of the most precious — as a 
place of phenomenological self-certification —*! of the senses, building an 
esoteric language based on deprivation.” However, this dispossession is 
like the casting down of a shield, which in the moment of perception of 
danger one wants to take up again: this happens when one of the girls gives 
in to the urge to take off her blindfold and has contact with the monster 
face to face. 

It is an impossible confrontation with a lying being, which declares 
itself human but whose otherness is revealed in its face, making it a tran- 
sient creature, which can assume a thousand forms. This is a “figure of 
absence” (Vernet 1988), a personification of fear and representation of evil 
like its many predecessors in the history of cinema; just think of the cult 
of Pennywise, IT, born from the pen of Stephen King (1986), which later 
became an iconic TV film in the 1990s (Wallace 1990) and more recently a 
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cinematographic bilogy (Muschietti 2017, 2019). Here the person respon- 
sible for the “coulrophobia” of an entire generation constructs fearfulness 
on mutations of the clownish semblance, which in the moment of revela- 
tion passes from affability to hunger, changing its mouth, eyes, propor- 
tions, and so on. Slender Man, an authentic case of bricolage, remix, and 
semiotic mash-up, also appropriates these elements, eventually closing the 
cycle when, in addition to terrorizing and capturing one by one the four 
protagonists who have evoked it, the entity also begins to haunt the inno- 
cent little sister of one of the victims, driving her insane. This madness, 
which leads to forced hospitalization (paratopic horror space par excel- 
lence)? and gruesome facial hallucinations of all sorts, will be expressed 
in a series of desperate screams: “He was faceless! He was faceless!” As 
demonstrated by the most typical of cinematographic horror procedures, 
the “jumpscare” (whose effect proves the importance of the face as a scopic 
device, which looks at us), the initial fright generated by a monstrous 
face that appears out of nowhere is usually immediately assimilated, and 
its terrifying potential declines (it is then the task of the diegesis and other 
formal solutions to keep it alive). On the contrary, if the face is absent, if 
there is only a head but the rest is missing, then the disturbance is perpetu- 
ated like an incessant an-epistemic horizon. As Trevor J. Blank and Lynne 
S. McNeill argue: “Fear has no face” (2018: 3). 


3. Ante litteram creepypastas: les sans-gueule 


In 1891 Marcel Schwob published Caeur double, a collection of fantasy 
and horror stories coming a few decades after those of Edgar Allan Poe 
but absolutely of the same stature. Among these, the short story of the 
Sans-gueule (literally, “The Without-snout”) stands out. It is a heart- 
breaking tale of two soldiers, found on the battlefield, physically alike 
and bearing the same wounds: deaf and blind, their faces blasted away 
by a howitzer. A doctor operates on them and gives them mouths which 
emit inarticulate sounds. A woman, a “quasi-widow” who is looking for 
her husband who she knows is missing and wounded, decides to take 
them both in because they remind her of her lost husband but then little 
by little begins to prefer one over the other, for reasons that have noth- 
ing to do with physical similarities, since the two bodies have lost the 
traits that made them significantly different (in the full Saussurian sense). 
Meanwhile, the other’s condition slowly worsens, and he eventually dies, 
throwing the woman into the despair of uncertainty: she does not know 
if the one who has left her was her husband or not while the survivor’s 
living body, totally unaware, continues to smoke from that slit it has in 
place of a mouth. 


118 Bruno Surace 


Here, therefore, the lost face is a perturbing object as a synecdoche of 
a dehumanization that coincides with a loss of sociality. The two men are 
no longer men but completely indecipherable physiologies. They also no 
longer have an identity. They breathe, require nourishment, and even seem 
to enjoy smoking, emitting strange gasps when they do so, but the lack of 
a face has made them something completely different, eliminating any pos- 
sibility of establishing even the simplest of communications, the phatic one, 
which enables us to understand if they can hear or comprehend. Whoever 
does not have a face is eventually in some way stripped, one by one, of 
Jakobson’s communication functions (1963), up to the meta-linguistic one. 
The face is thus configured as an interface, a linguistic bridge, between us 
and the other, which if eliminated makes any communicative exchange, 
any semiosic production, impossible. At most, the poor woman has to 
limit herself to interpreting these bodies, thus passing to a regime of sig- 
nification, without ever obtaining confirmation that they are possessed of 
any initial intentionality. Slender Man is equally frightening somatically 
because it lacks a lingua franca, a common metalanguage, an aid to under- 
standing its intentions. 

Schwob’s story, therefore, which is initially the narration of the horrors 
of war through a literary invention that actually reconstructs an episode 
not so infrequent at the time,” is also and above all a reflection on the 
facial device as a necessary threshold for the establishment of a meaning- 
ful relationship between us and otherness. The face is a precious sign. This 
exegesis is shared by a valuable, and very rare, semiotic study of the story: 


Structurally, the story pivots . . . on a description of the anxiety to which 
the “petite femme” is subject by reason of the need to choose between 
two featureless — not faces — but “surfaces”... . This passage makes it 
clear that her careful scrutiny does succeed in distinguishing something 
equivalent to a “face”... . But her anxiety stems from the impossibility 
of discerning a sign that would give one of the Sans-Gueule, but not the 
other, a face (the face of her husband): she cannot choose between them. 
So the problem is not to produce meaning by humanizing the faceless 
surfaces she is scanning; the problem is to opt for one rather than the 
other as being the “true” or “right” choice, the one that corresponds to 
her lost husband. The anxiety she displays in attempting to “read,” ... 
the undifferentiated text. .. . In short, the problematics of reading she is 
thus enacting derives from the classical conception of meaning as being 
unique and determinable and subject in consequence to acts of discern- 
ment as to the rightness or wrongness of specific “readings”: it is read- 
ing of the “readable,” not the scriptable. 

(Chambers 1984: 40)*° 
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And: 


At a number of points, the text is explicit that the two Sans-Gueules 
pose the problem of meaning. At the outset, they are a “double cicatrice 
arrondie, gigantesque et sans signification,” and later, “les deux coupes 
rouges couturées reposaient toujours sur les oreillers, avec cette méme 
absence de signification qui en faisaient une double énigme.” . . . Since 
it is the two Sans-Gueule together who form the mouth (even though 
each is equipped individually with a “palais beant” and a “tremblant 
moignon de langue”), the selectivity of love can only be self-defeating, 
and the production of meaning can only destroy the totality that offered 
the possibility of meaning. 

(Ibidem: 42) 


The rarefaction of facial features, therefore, from the perspective of a 
strong relationship between “soma” and “sema”,” coincides on the one 
hand with a loss of meaning and on the other with an almost inversely 
proportional increase in restlessness. Where the face empties, cognition 
fills with uncertainties and anxieties. In fact, Les Sans-gueule seems to pre- 
figure a sort of creepypasta literature, both in terms of the themes and 
rhetorical choices (the open ending, for example) and on account of its 
duration (it is a short, immediate story, which begins in medias res without 
getting lost in particular contextualizations). Of course, unlike creepypas- 
tas, it claims its own literary dignity, but what interests us is that it consti- 
tutes a cornerstone in a potential philology of the face/non-face as a device 
which elicits anxiety. 

The cinema will then become an ideal place for the development of this 
dimension of the contradicted face, clearly due to its media specificity based 
on the visual.” More or less marginal characters, but of great impact, with 
empty faces, will appear throughout the history of cinema. In the dream 
sequence in Spellbound (Hitchcock 1945), whose setting was masterfully 
designed by Salvador Dali, the character played by Gregory Peck relates a 
specific moment of his dream in which the manager of a gambling house 
introduces himself, fully dressed, but without a face. In one of old Isak 
Borg’s dreams in Smultronstället (Bergman 1957), a strange man appears, 
once again well dressed, whose face is a strange two-dimensional sur- 
face (as sometimes happens to faces in INLAND EMPIRE, Lynch 2006), 
white, with deformed and rarefied features, and eyes reduced to small slits. 
One of the witches in Polanski’s Macbeth (1971) is not only blind but the 
space where her eyes should be seems covered with skin as is also the case 
with the young Laura in Pieles (Casanova 2017), as if the eyes had never 
been there (therefore not a space of being that is no longer but a space 
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of nonbeing). In Joel Barish’s twisted dreams in Eternal Sunshine of the 
Spotless Mind (Gondry 2004), the progressive cancellation of Clementine’s 
memories coincides with the rarefaction of faces. There is a lengthy list of 
similar examples. 

What do all these characters have in common? They are, first of all, 
always relegated to a dream or hypnagogic dimension, designating one 
or other space or “allospace” (cf. Surace 2019). They are also marginal- 
ized. As signaling of the limen or threshold figures they seem to enjoy a 
limited cultural autonomy, which explodes in their perturbing power and 
immediately dissipates. The faceless, in other words, do not seem to have 
enough strength to hold up a whole narrative; they mostly act as props. 
This appears not to be true for some cases: one might argue that it is not 
the case with Slender Man or the Sans-gueule. But, in fact, Slender Man is 
rarely seen. Rather, it can be glimpsed from afar in the pareidolias, hinted 
at but not clearly shown, and even in the film based on this figure its pres- 
ence is marginal compared to the actions of the protagonists who strug- 
gle to escape it. When it appears it is either dark, blurred, or merely the 
means to an end (it does what it must do and disappears). Similarly, the 
Sans-gueule are the object of description in Schwob’s tale, while the story 
of the woman is its fulcrum. Most of the aforementioned faceless appear 
in dreams or magic as figures of passage, memory traces, residues of the 
unconscious, ultimately functional exclusively to the development of those 
who still have a face. In Eternal Sunshine of the Spotless Mind, the disap- 
pearing faces tell us about Joel’s memory loss, which we can see very well. 
In Spellbound the owner of the gambling house, representing the meton- 
ymy of the white cards thrown on the table, is a sign of the protagonist’s 
mental state. The “face” is therefore not only interface but also surface, on 
which traces of us are written — or in some cases erased — as demonstrated 
by other films, such as La Jetée (Marker 1962): “On a primary narrative 
level, as the protagonist’s mind, and the photogrammatic representation 
of his body, move between past, present and future, the only visual, rather 
than diegetic clues to temporal location are the repetitions of images, par- 
ticularly the face” (Chamarette 2012: 79). 

The empty face, the removed face, the thin face, or the rarefied face is a 
generous face in semiotic terms, which abandons the claim to mean some- 
thing for itself and begins to mean exclusively for the other, to be consumed 
quickly, and painfully. 


4. Les Yeux sans visage, Le Visage Sans Yeux 


In 1937 the character of The Blank appeared for the first time in the famous 
series of detective comics Dick Tracy: a very dangerous killer which is, 
again, faceless. Covered by a blindfold, the Blank’s face hides his identity, 
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although once “unmasked” he is revealed to be Frank Redrum (read back- 
ward and the word “murder” will appear, as anyone who has seen The 
Shining immediately realizes), a criminal with a horribly disfigured face, 
just like Erik, the Phantom of the Opera by Gaston Leroux,” or Spawn, 
another comic book character created in 1992 by Todd McFarlane, both 
committed to obscuring their disfigurement.*° 

These stories introduce us to a further dimension of the featureless face, 
similar in the disturbing effects elicited by its emptiness on those who see it 
but dissimilar in the order of the genesis of this emptiness. In these cases, 
the empty face is in fact not a non-face but a meta-face, placed on an 
“ante-face” (what remains of a face after an accident or what is behind the 
epidermis) that is to be concealed, somewhat in the manner of Leatherface 
in The Texas Chainsaw Massacre (Hooper 1974), who shows himself only 
with his face covered by the skin of his victims, thus wearing a full and 
empty mask at the same time: “Blank masks simultaneously erase identity 
and create spaces to project new meanings onto, prompting another dimen- 
sion to the visual iconography of horror film masks” (Heller-Nicholas 
2019: 112). 

These narratives deal with a face that has been lost. The loss is both 
physical and symbolic. Sometimes the subject feels that s/he has lost her/ 
his face symbolically and so covers her-/himself in white to cancel a miss- 
ing identity and seek shelter in a non-identity. Other times, however, it is a 
physical loss, as happens to Freddy Krueger in A Nightmare on Elm Street 
(Craven 1984), which sanctions the symbolic passage. In short, if soma is 
sema, it is also true that sema is soma. 

In Les Yeux sans visage (Franju 1960) this theme is developed through 
a triple semantization of the face. In the first instance, there is a discourse 
related to the living face and the dead face. Instead of ceasing its function, 
the latter becomes the possibility of re-facialization for those who have lost 
their faces, through futuristic transplants.*! The film places the emphasis in 
this case on the face as an obsession. In fact, the lost face must be regained, 
no matter at what price, even at the cost of sacrificing more than one per- 
son by luring them and taking their face away by force. The second dimen- 
sion of the face is that of the mask, which Christiane wears, for example, 
when she does not have an “other’s” face on her. Here we are dealing with 
a totally neutral mask, in fact, a white face, whose only significant ele- 
ments are the eyes, alive, which are behind it once worn. The gloomy non- 
expressiveness of the mask, also conveyed by other films such as Vanilla 
Sky (Crowe 2001), while on the one hand inducing a certain restlessness in 
the viewer, is at the same time a sign of a lack of acceptance by the wearer, 
rather as if it were a transitory device, useful only in the limbic waiting 
to regain possession of a new skin, as happens in Seconds (Frankeheimer 
1966). The third valorization of the face is that which passes from diegesis 
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to mimesis through the medium of film. In fact, the film not only stages a 
story in which the face is first lost and subsequently rediscovered, passing 
through the empty face of the mask but also emphasizes this facial obses- 
sion through a calibrated use of the foreground: 


It is almost impossible to read about the unnaturalness of the close-up 
and its association with death without thinking of the classical French 
horror film Les Yeux sans visage (1960), an adaptation of Jean Redon’s 
novel that demonstrates the notion of the close-up as a representation 
of “dismembered” body, a two-dimensional face severed from its body 
by the cinematic cut. The uniqueness of Franju’s film resides not neces- 
sarily in its use of the close-up (although it does that as well) but in the 
way it literalizes the notion regarding the monstrosity of the close-up in 
the story it tells. The question that it raises is the following: How does 
a film “attacking” one of the basic elements of engagement with the 
protagonist’s desire — the face of the actor — affect spectators regarding 
this very engagement? 

(Meiri and Kohen-Raz 2020: 48) 


Similar epidermal obsessions can be found in the aforementioned Pieles, a 
Spanish film with a grotesque flavor (not surprisingly a production by Alex 
de la Iglesia), in which a courageous operation is carried out concerning 
bodies and faces which are deformed for reasons as imaginative as they are 
realistic (that is, attributable to existing pathologies). Among the charac- 
ters can be found Laura who, as we mentioned earlier, has half a face and is 
eyeless (the place where the eyes should be is covered with skin); Samantha, 
who manifests a curious pathology whereby she has an anus instead of a 
mouth (and vice versa); Guile, whose face is completely burned; and Ana, 
whose left side of her face droops (her pathology is not specified in the 
film but could be hemifacial hyperplasia). If many of these cases are actu- 
ally provided with a face, albeit deviant, or as in the case of Guile with an 
ante-face (there is a very moving final scene in which he looks in the mir- 
ror after having undergone a maxillofacial operation, while his ex-beloved 
Ana proudly chooses to remain as she is), it is Laura’s case here that is 
most prominent, since hers is half a young and pretty face while, verti- 
cally, half a non-face. The girl’s sad story reveals how she has been locked 
up in a brothel all her life, in bondage since childhood to afford pleasure 
first to pedophiles and then when she grows up to women and men whose 
identity she does not know (being devoid of eyes). One of these gives her 
two diamonds - telling her: “The world is full of people it is better not to 
see . . . you deserve the most beautiful eyes in the world”- and puts them 
on her face like eyes. She will become so fond of them that she will no 
longer be able to part with them. The outcome of this sort of “symbolic 
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plastic” is again alienating. In the eyes of the beholder, or at least of those 
who watch the film, this face with its false ocularity is rather perturbing, 
reminding the viewer a little of the chilling button eyes in the world behind 
Coraline’s wall (Selik 2009). For Laura, however, these diamonds become 
indispensable, a form of re-appropriation of the self that passes through 
a specific somaticity. Only at the end, when she has found the love of an 
obese woman (who in the meantime has stolen her diamond eyes to pay off 
her debt), will she be able to accept her half-face and achieve happiness. 

Laura’s in Pieles is a face without eyes, while Christiane’s are eyes with- 
out a face. In both cases, a new look corresponds to a new attitude, as the 
claims of Bruiser, a 2000 film by George A. Romero, confirm. The director, 
in fact, after a career spent glorifying the emaciated and indistinguishable 
visages of zombies, this time depicts the killer Henry who once again opts 
for a white mask without features: two dot-like holes for eyes, a thin slit for 
a mouth, and a mere hint of a nose. The identity dimension is once again 
magnified through the empty face: “Through the blank mask, Henry’s 
monstrosity does not denote an eradication of his identity as such, but a 
transformation of it” (Heller-Nicholas 2019: 123). Similar are the charac- 
ters of The Invisible Man (H. G. Wells 1881), the last of which (filmed in 
2020 by Leigh Whannell) emphasizes the potential of the nonexistent face 
as a form of anonymity and “passport” for carrying out the worst possible 
atrocities. 

The emptying of the face therefore coincides with the cancellation of a 
series of stigmas that manifest themselves instead in the counterpart of the 
“full” face — abnormal, deformed, deviant, as in The Man Without a Face 
(Gibson 1993), in which the protagonist’s face is disfigured by burns, and, 
consequently, he is essentially an outcast, the character of John Hurt in The 
Elephant Man (Lynch 1980), or Roy in Mask (Bogdanovich 1985). 


5. Conclusions 


The figure of the “non-face” thus crosses many narrative spaces, from clas- 
sical to contemporary literature, from horror cinema to the grotesque, nat- 
urally also passing through animation if one thinks of the faceless demon 
in Spirited Away (Miyazaki 2001). It constitutes a degeneration of the face 
that configures an absent presence of a particularly perturbing character, 
which potentially magnifies the sociocultural specificities of which the face 
is a bearer: identity, idea of self, agency detection, aesthetic canons, and the 
notion of social and personal mask. 

The face/non-face induces hesitation precisely because it is devoid of these 
complex elements, thus leading to a dyscrasia — albeit perhaps a certain 
fascination, too — in the observer, as in the case of Slender Man but also 
posing as an ineffable heterotopia that recounts the feelings of those who 
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are sometimes forced to wear it, as in Pieles or Les Yeux sans visage. In both 
cases, however, this interpretation of the face/non-face is often entrusted 
to the receiver. It remains an enigmatic device, hovering at the edges of the 
semiosphere (and indeed the corpus of texts in which it appears is significant 
but restricted), a place where the face as a cultural construct is canceled 
and the humanity of the wearer inferred but not confirmed, as demonstrated 
by the Sans-gueule, machines of flesh that breathe but disturb because in 
them no specific intentionality can be perceived. On the other hand, this 
type of iconography, which also arises at times from tragic human events,” 
may culturally cover a specific semantic universe: that of monsters. 

It should be emphasized that the etymology of “monster” is that of the 
Latin “monére”, which means to “admonish” or warn, but there are also 
links to “show”* or make something visible. 


Monsters . . . are therefore not simple things or events but always 
require: a) to be recognized as such and b) to be interpreted. . .. How- 
ever, the same things may not always produce the same emotions, and 
this obviously also applies to the monsters proper, called to generate 
specific emotions, of a repulsive nature, as Benveniste has already told 
us, such as fright or terror. In order for these emotions to emerge, the 
manifestation in the phenomenal field of some deformity is necessary, 
which inevitably appeals to a canon of forms, and consequently to an 
idea of conformity. 

(Lancioni 2020: 84-85) 


The face/non-face cannot be beheld but comes into view as a “manifestation 
in the phenomenal field” relegating itself to it immediately relegates itself 
to the domain of the most irreducible otherness. That on the margins of the 
semiosphere — that of monstrosity, be it demonic, as in the case of Slender 
Man, or extraterrestrial, as for the “gray men” of A.I. Artificial Intelligence 
(Spielberg 2001), oblong like the monster of creepypasta tales, humanoid, 
with heads, but without any facial features. This monstrosity finds in the 
“empty” signifier of the empty face the springboard for a specific mean- 
ing, which is otherness. Thus, in the end, this peculiar facial dysmorphia 
transliterates from the level of expression to that of content, from a formal 
level to an ideological and political one, capable of recounting the anxieties 
and fears, but also prejudices and limits, of certain cultures.** Verily, just as 
there are cultures of the face, there are also cultures of the non-face. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 
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The story of the trial elicited great public interest and was extensively dealt with 
in both the American and international press. See www.latimes.com/nation/ 
nationnow/la-na-nn-slenderman-girls-adult-case-20150810-story.html (last ac- 
cessed 05 January 2021). 

My translation from Portuguese. 

I reflected on this peculiar connection between the imaginary and the real in 
Surace (2020b) unicorni. 

We limit ourselves to suggesting that since it is not an act committed by a 
single person but rather a “shared brutality”, the social role assumed by the 
imaginary linked to Slender Man is relevant enough to warrant dealing with the 
matter not only exclusively in psychiatric terms but also in semiotic terms. 
This is not the first case, and even in the pre-digital world episodes can be 
traced in which a certain maniacality deriving from fictional imaginings was 
then dangerously transferred into reality. In this relation we consider the bur- 
geoning field of criminal humanities to be of great importance (see Arntfield 
and Danesi 2016). 

“an emergent horror genre that manifests through the form of digital fiction, 
characterized by unsettling paranormal and horror content copied, pasted and 
remixed on social media and Web 2.0 platforms under the guise of real and 
lived encounters” (Ondrak 2018: 162). See also Chess and Newsom 2015. 

See Page 2018; Henriksen 2018. 

The concepts of author and model reader are found in Eco 1979. 

On the notion of narrative ecosystem, see Pescatore 2018. 

The notions of “natural narrative” and its opposite “artificial narrative” are in 
Eco 1994, and respectively designate a text’s claim to be read as reporting truth 
(as in the case of a news program but also of an urban legend) or fiction (as in 
the case of a fantasy or science fiction novel). That a narrative presents itself 
as natural or artificial is not an explicit fact but has to do with the strategies of 
drafting the text itself. 

The reference is to Saler 2012 who “has discussed the capacity of certain works 
of fiction to effectively transcend their own fictionality, to create imaginary 
worlds that become, through deliberate inhabitation by their audiences, virtual 
worlds” (Tolbert 2015: 50). 

Cf. Magilow et al. 2012; Fedorov 2018. 

Cf. Peck 2022. 

I investigated this concept in Surace (2021b) with specific reference to artificial 
and robotic faces. 

Cf. Greimas 1966. 

We refer to it as a myth, naturally in terms of the mythological reading of soci- 
ety proposed in Barthes 1957, but also relying on the idea of contemporaneity 
as founded on “low-intensity myths” (that is, in short, less powerful and persis- 
tent, more ephemeral and transient) in Ortoleva 2019. 

On the phenomena of online virality see, firstly, Marino and Thibault 2016, for 
a compendium. An initial but good definition, which is the one underlying our 
use of the term, is the following: “First of all, we have to point out that we are 
dealing with two different forms of virality, which may be complementary but 
nevertheless need to be kept theoretically separated: the first one entails a piece 
of media content spreading pervasively and the other entails the practice of cre- 
ating other contents from a first one understood as the model or prototype. In 
the latter case we have a token that establishes a type from which other tokens 
are created by means of replication and modification; this is what happens with 
memes, which we may conceive, with a pun, as a form of ‘complex virality’ (as 
opposed to ‘simple virality’). As suggested by Jenkins, Ford, and Green (2013), 
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we may articulate the opposition between an old and a new model of content 
use: the first one, called stickiness, defines when many people in one place are 
enjoying a given content (as in the case of a successful article or website) and 
the latter, called spreadability, applies when one content is placed almost liter- 
ally everywhere for everybody to peruse with ease (as in the case of a viral pic- 
ture or video or of a series of memes with the same base). What we call virality, 
including both types described above, owes as much to the replicability as to 
the customization allowed by digital technologies; it is not merely an issue of 
copying a given content, but rather of adapting, appropriating and properly 
translating a given content according to need” (Marino 2022). 

It is evident that, in this case, the COVID-19 pandemic, with its overbearing 
remodeling of the global imagination and the reconfiguration of common prox- 
emics, has also played a fundamental role and perhaps above all in the way our 
faces mean outside and inside the digital context. See Leone 2020. 

On this point, it is possible to approach it with a more psychoanalytical gaze, 
for example, Lacan 1991, and philosophical, Zizek 2000. 

As proof of this, just think of the great and lasting (so much so that it has its 
roots in the origins of the web) success of online videos that prove conspiracy 
theories visually. The basic idea is that “sightings” or in any case visual 
evidence are more effective than other types of evidence. A video showing 
aliens in one’s backyard will receive more hits than an alleged declassified 
document from any government in which evidence of the same fact is set out 
in writing. 

And in fact, if one thinks about it, one can “lose” face, thus effectively losing 
one’s reputation (and this is a fact shared culturally from West to East, so much 
so that in China a common insult is to say that someone “has no face”), or one 
“wants to lose face” to protect oneself even from embarrassment as in the case 
of the “facepalm”, a veiling of the face analyzed in Marino 2020. 

The notion of paratopic space is taken from Greimas’ theory of the relationship 
between space and narrativity. Specifically, the paratopic space is understood 
as the space in which certain skills are acquired by the characters in the story. 
See Greimas 1976. 

See Surace 2022: 78. 

It was not, nor is it unfortunately, merely a literary invention. War wounds 
have always disfigured thousands of faces. As regards the twentieth-century 
wars, here you can find photo albums of plastic surgery practiced at the 
King George Military Hospital in London https://wellcomelibrary.org/item/ 
b20160999#?m=0&cv=0&c=0&s=0; regarding more recent wars an inter- 
esting interview with Col. Robert G. Hale, commander of the Army’s dental 
and trauma research detachment at Fort Sam Houston in San Antonio for 
the New York Times, can be found at www.nytimes.com/2013/12/03/science/ 
healing-soldiers-most-exposed-wounds.html?pagewanted=all (last accessed 05 
January 2021). 

Here specifically Chambers reflects on the following passage of the story: “Elle 
allait éternellement de Pun a Pautre, épiant une indication, attendant un signe. 
Elle guettait ces surfaces rouges qui ne bougeraient jamais plus. Elle regardait 
avec anxiété ces énormes cicatrices dont elle distinguait graduellement les cou- 
tures comme on connait les traits des visages aimés. Elle les examinait tour a 
tour, ainsi que l’on considère les épreuves d'une photographie, sans se décider a 
choisir.” 
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The fertile soma/sema formula has a long philosophical history that dates back 
to Plato’s Gorgias and goes as far as the post-structuralist works of Derrida 
1972 or the semiotic works of Fontanille 2004. 

On the importance of the face in cinema, from its origins, see Jandelli 2016. 
Relevant in this sense is the question that Strong puts at the beginning of his 
bioethical reflection on facial transplantation: “If the Phantom of the Opera 
were offered a face transplant, would he have the capacity to say ‘no,’ or would 
he be so desperate for a new face that he would grasp at straws?” (2004: 13). 
On the relationship between body image and disfigurement, see Rumsey and 
Harcourt 2004. 

Today, face transplantation is a scientific reality, also thanks to the pioneering 
work (not only in the medical field but also in the bioethical field) conducted 
by researchers such as Maria Siemionow. See her 2019 volume in which she 
reconstructs the stages, including the philosophical ones that led to her first face 
transplant. 

The notion was codified by Jurij Lotman. See Lotman 1984. For a good defini- 
tion see Cobley 2009. 

Proof of this are precisely those people who, for example, victims of an acci- 
dent, lose their faces and need a transplant or cases in the news such as that of 
the child born in Portugal without facial features: www.bbc.com/news/world- 
europe-50166857 (last accessed 05 January 2021). 

See Benveniste 1969. 

My translation from Italian. 

On these topics see also Surace (2020a) culture. 
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8 Imaginary faces 


Aliens, monsters, and otherness! 


Remo Gramigna 


1. The significance of the human face 


The fascination with the study of the human face is constant in the history 
of humanity. While the face is a visible element of the anatomy and appear- 
ance of the human species, it remains quite an enigmatic subject and very 
difficult to fathom. From ancient treatises on physiognomy to the ubiquity 
of automated face detection (Kosinski et al. 2023) and AI face recognition 
systems much in vogue in today’s “culture of surveillance” (Gates 2011), 
the face remains a subject of profound interest that cuts across a range of 
disciplines, from evolutionary biology to artificial intelligence. 

Jurgen Ruesch (1956) refers to the human face as the signifier par excel- 
lence, Ray Birdwhistell (1970) as a message board, and Max Thorek 
(1946) as an advertisement to the world. Indeed, faces communicate. They 
always tell a story, for they represent the visible core of people’s iden- 
tity. As research carried out in cognitive psychology and neuroscience has 
shown, the face is a broadcast screen onto which a multilayered array of 
facial stimuli is elicited and constantly monitored by others in real-life set- 
tings. Information such as gender, identity, age, skin pigmentations, and 
basic emotions are inferred and “read” from faces. Darwin (1872) held 
that facial expressions in humans have something in common with other 
animals. Other thinkers sought to identify a possible grammar of emotions 
through the study of facial expressions (Le Brun 1992). 

The face is the site of perception, signs, and semiosis. It is quite reveal- 
ing to notice how biology condensed four out of five senses in the human 
face. Indeed, sight, hearing, smell, and taste are all grouped in the upper 
part of the body. Faces are also socially and culturally significant as they 
give a sense of self-identity and the template for enabling recognition by 
others - who we are and where we came from. As Juri Lotman pointed out 
in Culture and Explosion (2009): 


Man became man when he realized himself as a man. And this occurred 
when he noticed that the different members of the human race consisted 
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of different people, different voices and different experiences. The face 
of the individual, as with individual sexual selection, was probably the 
first invention of man as a man. 

(Lotman 2009: 155) 


The central role and the significance of the human face as a scientific sub- 
ject are, thus, evident both in the so-called exact or hard sciences, such as 
medicine, evolutionary biology, and genetics, and in the sciences of man 
and society, such as cultural anthropology, history, literature, and psy- 
chology. While the exact sciences deal with the human face in its organic 
dimension, the study of its physiological and biological functions, its ori- 
gin, evolution, and the genetic architecture that lies beneath it. The human 
sciences deal with the social functions and values as well as the cultural 
and societal implications of the study of the human face. Each culture 
elaborates a specific set of values that tends to vary in space and time. The 
human face, coupled with its many representations and the social values 
attached to them, constitutes an integral part of it. 

The sciences of man and society study the human face generally based on 
‘representations’ that culture itself filters, stores, and hands down within 
its social imaginary and its own tradition, namely their own “self-portrait” 
of a culture (Uspenskij et al. 1973; Lotman 1979).? Texts, in both oral and 
written form, are illustrations of such representations. Portraits and self- 
portraits, drawings, caricatures, statues, and pictures make up some of the 
visual repertoires of face representations as archaeological history, art his- 
tory, and the history of artifacts have been well documented. 

Today, this array of face representations has been boosted by the prolifera- 
tion of digital representations of the human face. It suffices to mention the 
widespread use of the “selfie” by the new generations (Peraica 2017; Rottberg 
2014) and the wide access to photo production and reproduction on a mass 
scale (Fontcuberta 2016). Personal testimonies and first-hand accounts of the 
members of a society - the interpreters - are also pivotal for reconstructing the 
history and the study of visual and/or verbal representations of the face. Given 
the centrality and significance of this theme, it is not surprising to find a wide 
range of depictions of human faces in many cultures and societies around the 
world, ranging from the earliest findings of cave paintings from the Paleo- 
lithic period to the digital or “artificial faces” (Leone 2021d) of the Internet 
age generated through computer algorithms and digital manipulation. 


2. Looking at faces beyond normativity: on monstrosity and 
wonder 


Yet a question remains as to whether only humans and primates are 
endowed with face-like traits or also nonhuman animals (Leone 2021d: 2) 
or if even extraterrestrial beings have a face. Undoubtedly, there seems to be 
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a conundrum about the study of the representation of nonhuman faces, not 
so much in regard to nonhuman animals but in respect to off-planet entities 
or what goes under the rubric of “post-human” (Graham 2002). Whether it 
is possible to conceive of faces that are not human in this specific sense of the 
term remains an open question. Are there faces that are literally out of this 
world? Do gods and aliens have faces? And if so, what do they look like? 

Indeed, it should be noted that, besides the representations of men’s faces 
in their manifold forms and shapes, there are numerous depictions and 
descriptions of nonhuman or pseudo-human faces, as it were. These are, 
very often, representations of gods and goddesses, imaginary beings, and 
fantastic creatures that often have a very ancient pedigree and whose origin 
is very difficult to trace with precision because they are shrouded in myths, 
archetypes, fiction, and legends (Borges 1957). Ancient teratology is dot- 
ted with descriptions of the physical appearance of monstrous, shapeless, 
horrible, and evil beings that could be grouped in a generic fashion, in the 
category of ‘artificial faces’, ‘deviant’, or ‘aberrant faces’ in order to set 
them apart from the mere ‘organic faces.’ 

Internationally renowned scholars, such as Umberto Eco (2007) and, 
much earlier, the philosopher Karl Rosenkranz (1853), have traced the his- 
tory of the “aesthetics of ugliness” in the social imaginary of Western socie- 
ties and have pointed out that the concepts of the ‘stranger’, the ‘monster’, 
and the ‘alien’ often overlap. This point is worth pondering, and I will 
come back to it. The concept of monstrum, in its mutability, sums up all 
the aberrant forms and arises as a general category in which to include 
what deviates from the norm: faces that go beyond normativity. Indeed, 
according to the Latin lexicographer Festus, things that come out of the 
natural world fall into the genre of monstra. 

Monstrum comes from the Latin m6néo (to admonish, to warn). Mon- 
strum was a polysemous term. It primarily designated divine warnings, 
which burst forth unexpectedly, almost assaulting us. It meant, then, ‘mon- 
ster’ in our contemporary sense, and finally, it could also be used as a 
synonym for ‘wonder’. In its original meaning, monstrum is, thus, the sud- 
den appearance of something extraordinary, which violates nature and is a 
warning and a caution to man. 

This term opens up a chasm of meaning, and the etymological trace here 
becomes the cue to identify the link established between monstrosity and 
wonder. The monster bears witness to unprecedented, unforeseen, ‘subver- 
sive’ forms of life in a certain sense, which, by producing wonder, induce 
reflection and the redefinition of our common sense and the solid reality to 
which it clings (Cattaneo 2002: 42-43). As Alberto Abruzzese pointed out, 
the monster represents 


a figure that is distinguished by the imaginative force with which its 
forms upset the ordinary perception of nature and men, by the fear it 
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instills in them of their chaotic, alien, strange character. A formless crea- 
ture. A hybrid figure, often between the human and the beastly, in which 
a diabolical split between soul and flesh, spirit and matter, excludes any 
harmony, any veiled beauty. 

(Abruzzese 2003: 385)* 


As a ‘formless form’, the monstrum stirs our imagination, making us con- 
verse with the unknown toward the abyss of a perceptive emotion based 
on wonder. 


3. Outer appearance and face representations of aliens 


A study of the outer appearance of alien races and their representations 
is, in and of itself, a very fascinating as well as a thought-provoking sub- 
ject. This chapter is the result of an attempt to penetrate this more or less 
unexplored territory. Undoubtedly, it is not very common for academics to 
discuss this particular subject, although scientists and scholars are catching 
up in covering some aspects of this large and controversial subject (Cen- 
tini et al. 1998; Jung 1967, 1978; Lewis and Kahn 2005; Battaglia 2005; 
Roth 2005; Wegner 2003). This makes the topic under examination even 
more compelling. The theme of the existence of ET races has a fairly long 
history (Dick 1980). Already from the beginning of the 1950s, there was 
a rise in public awareness about flying saucers and unidentified objects, as 
evidenced in the press coverage of such phenomena in the United States 
(Strentz 1970). The literature on the subject is vast. It encompasses studies 
in the fields of ufology, cultural anthropology, critical studies, psychia- 
try, psychology (Newman and Baumeister 1996; Blackmore 1999), par- 
apsychology, psychic research (Swann 2017, 2018), and occult studies, 
social sciences, as well as cinema, science fiction studies, and narratology 
(Lepselter 2005, 2016).° 

Today, the subject has seen a resurfacing. This is partially due to the 
encapsulating of an element of “alien disclosure”, “alien abduction”, and 
contacts with extraterrestrial civilizations within the theories that are gen- 
erally grouped under the rubric of “conspiracy theories” (Aaronovitch 
2009; Brotherton 2015; Gramigna 2021; Wheen 2004; Thompson 2013). 
Academic studies carried out on this subject have generally focused on 
this one aspect, namely the conspiratorial element embedded in such dis- 
courses. As it will be apparent in what follows, however, I will take a dif- 
ferent route. For our present purposes, we shall be concerned primarily 
with the representation of alien races, with a specific focus on the face and 
the outer appearance, and leave aside the rest of it as it will fall outside the 
scope of the present inquiry.° 
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To provide the reader with a quick reference of what I refer to as the 
outer appearance of an alien race, we may consider the visual represen- 
tation of the type of alien generally referred to as the “Greys”. Visual 
representations of the Greys are often present in numerous accounts of 
ET encounters as well as in fictional works.” The image of the Grey has 
become a sort of quintessential visual icon of what an alien looks like in 
the social imagery of Western culture. Suchlike visual representations have 
become ingrained in the social perception of what the representation of the 
physical appearance of an alien looks like. Greys are typically purported as 
humanoid beings with gray skin, elongated bodies, very large heads, and 
very large black eyes. Greys are often depicted as having 


[a] hairless body that is gray or green in color, have very large almond- 
shaped eyes, very thin body and a large head again with no hairs, no 
eyebrows, or eyelashes, a very flat nose and ears or sometimes just holes 


in place of ears. 
(Challoner 2005) 


Such representations, as the one just described, surface in the culture 
so much so that they have a life of their own. Borrowing a term coined 
by Susan Blackmore (1999), these images make up what the author has 
termed “memeplexes”, that is, ideas that grow, spread, and develop along 
the lines of biological evolution in a cultural system. These are the kinds 
of representations this chapter focuses on. It is also worth noting that the 
image of the Grey as a prototypical alien representation is also evident in 
the emoticons of the ‘alien’ available on many social media platforms, from 
Facebook to Twitter, as well as in all instant messaging apps. 

My interest in this particular area of research, thus, lies in one particular 
aspect of this large and complex issue. The focus of the present chapter 
revolves around the physical descriptions of alien races that can be found 
around the very broad discussions on UFOs, alien races, off-planet enti- 
ties, and ET encounters. Indeed, descriptions devoted to the explanations 
and discussions of the outer appearance of extraterrestrials abound in the 
literature on the subject. However, it is hard to find a systematic and coher- 
ent survey that catalogs with scientific rigor and accuracy the forms of 
extraterrestrials based on their physical appearance. This would definitely 
be the work of a lifetime and, as far as I am concerned, no one has success- 
fully achieved such a project up to now. Undoubtedly, I am fully aware that 
the present project only scratches the surface of such a large and complex 
phenomenon and is far from being exhaustive. 

To my knowledge, there has not yet been a study that has cataloged all 
alien species according to their outer appearance. The closest you can get 
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is the survey of alien races titled The Alien Races Book (Carlson 2017), 
which contains a very large number of images and descriptions of alien 
races. Other attempts at cataloging different types of aliens can be found 
in Centini et al. (1998), who provide a classification of twenty-one types 
of aliens and include depictions and sketches of the types discussed. The 
authors refer to an earlier study by the Brazilian Jaser Pereira, who has 
cataloged 333 types of “animated entities” (Centini et al. 1998: 79). Many 
other descriptions, typologies, and classifications can be found on YouTube 
channels although these attempts are unsystematic.® 

Indeed, information on the subject is scattered throughout many dif- 
ferent sources: personal accounts and first-hand reports rendered public 
by ET experiencers, magazines and journal articles, books and literature 
of various sorts — from science fiction to WikiLeaks reports — mainstream 
media as well as “alternative” media outlets, videos, and podcasts, not to 
mention the cultural myths, legends, stories, and rumors circulating on this 
subject. This makes the job of a researcher a challenging one, at best. 


4. What do aliens look like? A physiognomy of aliens 


As there is a physiognomy of man, there is a physiognomy of aliens. This 
is an interesting hypothesis that is worth exploring. Ancient physiognomy 
has a long pedigree. It has been used since time immemorial in the West — 
from Aristotle up to Cesare Lombroso — and in the East, where it flourished 
under the aegis of Fakiir Al-Din Al-Razi. Although the face is a visible ele- 
ment of man’s anatomy and outward appearance, it still remains an enig- 
matic and difficult-to-read object. It is no coincidence that the study of the 
face has given rise to multiple cultural models and various interpretations, 
schemes, and modes of representation. Ancient physiognomy, in fact, pos- 
tulated a tight link between two opposite poles, the inner (invisible) and 
outer (visible) appearance, thus identifying a close connection between the 
two, that is, the principle of identity between inside and outside, being and 
appearing, face and character. This aspect is quite evident in Jhoann Kas- 
par Lavater’s definition of physiognomy. Physiognomy means the ability 
to recognize, from a man’s outward appearance, his inwardness (Lavater 
1989: 31). Thus, the challenge and aim, often disregarded, of traditional 
physiognomy as a pseudo-scientific project has been, since ancient times, 
to enclose the human face within certain categories, specific types, codes, 
characters, patterns, and so on. This desire to determine the face, typical of 
all physiognomy, constitutes its profound ambition and greatest limitation. 

It is well known that Aristotle, in Prior Analytics, gave an example of 
how to use certain types of signs to infer and identify the character of 
people. For instance, the large extremities of the lion are regarded, from 
this standpoint, as an indicator of courage. Physiognomy is generally 
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regarded as a pseudo-science whose purpose is to infer personality traits 
from the physical appearance of an individual. Physiognomy was predi- 
cated upon a connection between the inner, invisible characteristics of 
man from the outer, visible features of physiognomic traits (Antonini 
1900; Caroli 1995; Daston and Park 1998; Delaunay 1928; Foerster 
1884, 1893). Traditional physiognomy is based on the principle of iden- 
tity. This principle postulates that there is a correspondence between 
being and appearance, exteriority and interiority, face and character, and 
body and soul. It is, evidently, a rigid and deterministic scheme that con- 
tributes to making the face something rigid, pigeonholing it into neat 
and predetermined dichotomies: face and mask, appearance and essence, 
expression and reticence, zoomorphic traits, and human features. This 
type of approach, based on the principle of identity, held sway from Aris- 
totle until the nineteenth century. 

As pointed out before, physiognomic descriptions of what aliens look 
like are not a reminiscence of a science fiction movie but are very much pre- 
sent in first-hand accounts of those who reported having had interaction 
and/or communication with such entities. Thus, there is a physiognomy 
of man, and there is a physiognomy of aliens. The latter is applicable and 
refers to the characterization of the physical appearance of alien races. In 
this respect, it follows the same logic as classic physiognomy, namely to 
identify and sort out alien types from their outer appearance. Such descrip- 
tions provide an identikit for the identification and recognition of alien spe- 
cies and discerning between them. There are also key differences between 
the two as there is no intention to infer personality traits from the physical 
appearance of aliens but only to identify, cluster, and categorize the aliens 
encountered into different groups or families. 

This is not surprising, considering that attempts at cataloging mon- 
strous creatures and evil entities of all sorts are not a novelty, especially in 
the domain of aesthetics and art history and concerning specific periods 
of Western history, such as the Middle Ages (Baltrušaitis 1955, 1960; 
Kappler 1980). Furthermore, it is well known from research carried out 
in the fields of forensic anthropology, forensic medicine, and forensic 
genetics that personal identification of people is based on several criteria 
and methods, among which is found in the physiognomic study of the 
face (Falco 1923). Personal identification is based on different types of 
representations of the face of the individual, photographs being the most 
obvious ones. To the image of the face, one must add other types of rep- 
resentations: sketches, parodies, drawings, and oral and written descrip- 
tions. A similar procedure is used by experts in the field of ufology to 
identify alien races. 

What struck me is that descriptions of aliens often dwell not only on 
the characterization of the shape and size of the body, skin colors, hands, 
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and fingers but also on a description of facial features, for instance, the 
shape and color of the eyes, shape of the head and the ears, and so forth. 
May this be a proclivity of the human eye, it is significant that faces play a 
significant role in descriptions of alien races, too, as they are a key element 
in identifying people’s identities. In what follows, I will examine two main 
areas to expound my study. Firstly, I will dwell on a brief discussion on 
the concept of ‘representation’. I argue that this concept is a useful tool for 
the analyst of culture inasmuch as what we are dealing with when studying 
faces are usually representations of facial features embedded in visual or 
oral records. Thus, a study of the face from a cultural point of view entails 
a study of such representations. This is a key concept for unpacking this 
subject. 

Secondly, I will provide an overview of those accounts of the people who 
have witnessed or claim to have had interactions with off-planet entities. 
These unique individuals are generally referred to as “alien abductees” in 
the literature on ufology as well as in “alternative” media or as “alien 
abduction”. I will, instead, use slightly different terminology and refer to 
the subject who had interactions and communication with alien races as 
“experiencers”. This choice of terminology is predicated upon the idea that 
the term “abduction” retains a somewhat negative connotation, while the 
term “experiencer” is a more neutral word. When an “alien experiencer” 
has reported his own experience first-hand, he or she can be regarded as an 
interpreter of the alien culture he or she knows. 

At this juncture, it is also worth mentioning that cultural anthropol- 
ogy, ethnography, and folklore studies have for centuries investigated 
phenomena of the unnatural, the supernatural, the paranormal, and the 
mystical by relying on oral or written testimonies derived from the “inter- 
preters” themselves, namely those enmeshed in a particular set of systems 
and believes (Kluckhon 1949; Rosaldo 1980) that witnessed significant 
events in a specific sociocultural setting. Think, for instance, of the vast 
amount of research conducted on the culture of shamans, shamanism, and 
medicine men (Castaneda 1974; Mails 1979). 

Let me also clarify, from the outset, what my stance is in regard to the 
subject treated. Despite the term “conspiracy theory” being very much in 
fashion, it is not my intention to couch the views discussed in this chapter 
from this point of view, nor is my contention to attach a value judgment 
to the theory discussed. My goal is fairly humble. That is, to show that the 
relevance of the face as an indicator of identity and recognition is found 
not only in the identification of people but also in the description of alien 
races provided by the witnesses who have made their experiences public. 
Thus, I want to make clear that my interest in the theories is primarily in 
interpreting the work of alien experiencers as an iconic representation of 
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a ubiquitous and global phenomenon — undoubtedly, a phenomenon that 
today has reached a global scale. 


5. The nature and paradox of representations 


As said earlier, the sciences of man and society study the human face on the 
basis of ‘representations’ that culture itself has handed down in its social 
imaginary. Thus, a discussion on the concept of representation is in order. 
How can such representations be studied? What is their use? To start with, 
the approach to the study of representations stemming from semiotics — the 
study of signs, semiosis, and meaning-making — should be able to distin- 
guish various systems of representations. The range of representations var- 
ies. It includes verbal representations, such as oral and written language, 
as well as visual data, such as diagrams, drawings, photos, and so on. 
Furthermore, from a semiotic perspective, it is worth discerning between 
different types of systems of representation and providing a comparative 
analysis of their grammar. Visual representations of the human face are, 
thus, multifarious. It goes without saying that visual representations of the 
face play a key role in the process of recognition of others (Gramigna and 
Voto 2021). 

The distinction between analogical and digital information and codifica- 
tion is an important corollary to this discussion. This distinction applies to 
the representation of faces, too, and should not be disregarded. This yields 
the distinction between analogical and digital representations of faces, a 
difference that often goes unnoticed. When we discuss organic faces as well 
as when we talk about the human face in concrete interactions and every- 
day life settings, we operate in the realm of analogical codification. The lat- 
ter “constitutes a series of symbols that in their proportions and relations 
are similar to the thing, idea, or event for which they stand” (Ruesch and 
Kees 1956: 8). While analogical codification deals with continuous func- 
tions, digital codification is based on discrete step intervals. The organic 
face, as embedded in the network of nonverbal communication, is an ana- 
logic codification device. This is an aspect worth pondering, and such dis- 
tinction should not be eliminated. 

The concept of “representation” is relevant to the present discussion. 
This is a complex concept for which there is no universal definition. Each 
discipline has provided its own meaning to it. In his Tractatus Logico- 
Philosophicus, Wittgenstein provided more than fifteen definitions to 
explain what the concept of representation stands for. The verbs ‘to repre- 
sent’, ‘to present’, ‘to express’, ‘to exhibit’, ‘to reflect’, ‘to project’, and ‘to 
keep in place/to be in place of’ are some of these terms. What is important 
to stress is that in every form of representation, something is in the place 
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of something else. This is one of the most important key aspects of every 
representation. To represent means to stand for something else, which is 
at one and the same time evoked and erased. If we take a particular exam- 
ple of a representation, for instance, an ID card, one would say that this 
is an illustration of a representation of a human face. Usually in any ID 
card, there is a pictorial representation of the human face. However, as 
said before, a representation entails two aspects: (i) to point to a particular 
person, object, or event and (ii) the real person, however, is erased from 
the representation because he is not physically present there. This is the 
paradox of the representation, that is, to evoke or call forth the object 
represented and, at one and the same time, to erase it. We can, therefore, 
conclude that the principle of duplication is at the base of the nature of 
representation (Gil 1980). 


6. Defining aliens: a real conundrum 


There is a real conundrum in this field of research. Because reports come 
from very different sources, it can be challenging to disentangle the infor- 
mation provided and to assess and ascertain the accuracy and reliability 
of these sources. Moreover, fictional books and film industries — especially 
Hollywood — have drawn on the topos of the alien, which is often present 
as an element or a character of a science fiction plot with certain stereo- 
typical features. This has yielded a mix of facts and fiction which is not so 
easy to disentangle. This has created a plethora of texts, myths, and legends 
surfacing in different cultures in different channels and forms — from Inter- 
net memes to YouTube videos — and variations that are sedimented in the 
“social imagery” (Taylor 2003) and the collective consciousness of people. 

If this is not enough to contend with, it is worth recalling that perception 
and anthropomorphism are interlocked phenomena, a link that has been 
aptly emphasized and discussed at length in the study Faces in the Clouds 
(1995) by Stewart E. Guthrie. As can be gleaned from this study, the pro- 
clivity to perceive humanlike forms is a ubiquitous phenomenon: 


Faces and other human forms seem to pop out at us on all sides. Chance 
images in clouds, landforms, and ink blots present eyes, profiles, or 
whole figures. Voices murmur or whisper in the wind and waves. We see 
the world not only as alive but also as humanlike. Anthropomorphism 
pervades our thought and action. 

(Guthrie 1995: 62) 


Thus, it is probably a natural predisposition of the human brain to gauge 
inputs from the external world regarding recognizable patterns. And the 
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face is undoubtedly one of the most recognizable shapes. Thus, some of the 
descriptions of encounters with aliens that we know of dwell on the face of 
the being encountered, and there is often an element of anthropomorphism 
attached to it. This aspect caught my attention. 

Because some of the accounts of ET encounters provide some clues and 
give some attention to the representation of what may be referred to as 
“alien face” — namely, what an off-planet entity looks like to the human 
eye — then it is relevant to set out an inquiry along the lines of the pre- 
sent study and bring up the testimony of those who have experienced and 
described their experience first-hand. However, this terminology would 
immediately set aside “alien face” instead of “human face” in a dicho- 
tomic fashion. At this conjunction, a note of caution should be sounded. 
It is perhaps inaccurate to refer to the outer appearance of an off-planet 
entity as an “alien face”. Undoubtedly, the organic face has its biological 
roots and functions. Thus, the face, in its narrow meaning, is “an assembly 
of bone, gristle, muscle, skin, hair, blood, blood vessels, and organic mat- 
ter”. Its physical function is “to present the chief organs of the senses to 
the world and to operate them with the utmost advantage” (Brophy 1945: 
12-13). However, the term “face” has been used prescinding from its nar- 
row meaning — the organic face — in order to make sense of experiences 
and go beyond the face as a physical organ. Think of the perception of the 
environment or the morphology of urban space in terms of a face — Sim- 
mel’s morphology or “face of the city”. Likewise, one can describe the 
face of a building, the “face of history”, and many more examples of this 
logic can be added up. Thus, besides a narrow definition of a face in its 
biological dimension, there is also a wider meaning of the face understood 
as a broader phenomenon, and that includes the outer appearance given by 
facial characteristics. In this regard, the term ‘face’ can be used to discuss 
the outer appearance of what goes beyond being human. 

One may immediately react to the argument of this chapter with a mix 
of suspicion and incredulity. Indeed, some might ask whether extraterres- 
trials have faces or whether they exist at all. Is it even thinkable to take up 
this subject as a topic of research? To some, this may seem rather bizarre. 
However, in this chapter, I argue that asking these questions is relevant 
inasmuch as one cannot simply disregard the issue as irrational, illogical, 
implausible, mistaken, or simply impossible. On the contrary, there is a 
host of pivotal questions that arise from taking up this subject and that are 
worth exploring. Some of these issues fall outside the scope of the present 
chapter. It is, nonetheless, worth mentioning them in order to fathom the 
depth and significance of the issue at stake. The themes of make-believe 
and degrees or gradients of credulity (what is believable and what is not), 
the social construction of reality, simulation, and the question of veracity 
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and truthfulness through discourse, the issue of information and disinfor- 
mation, the question of culture, counter-culture, and exo-culture, the prob- 
lems of censorship and free speech are all relevant themes that make up an 
important corollary to this subject which cannot be disposed of in a few 
words and would deserve a separate and much lengthier elaboration. 


7. Constructing otherness: the alien/human dichotomy and the 
construction of the ‘us’ versus ‘them’ narrative 


From the viewpoint of the study of cultural phenomena, what we are deal- 
ing with are ‘texts’ of various nature — understood in the broad sense of 
the term as to encompass anything that has meaning — which by all means 
makes up the texture of a system. A broad definition of text encompasses 


not only written texts, literary in particular but any coherent verbal 
statement, oral, too. “Text” can even designate a vehicle of articulate 
global signification: painting, theater representation, dance, ritual. 
Finally, and this is the maximum extension, culture — besides textual- 
ized cultures — can be thought of itself as a text. 

(Segre 1982: 677) 


Ultimately, what we are dealing with are “cultural units”, that is, “simply 
anything that is culturally defined and distinguished as an entity. It may be 
a person, place, thing, feeling, state of affairs, sense of foreboding, fantasy, 
hallucination, hope or idea” (Eco 1976: 67). 

Although some texts may be regarded as “non-texts” from the point of 
view of a specific culture and relegated to the fringes of cultures, this does 
not mean that they lose their status as texts and have no value at all. On 
the contrary, as much of the research in the field of the semiotics of cul- 
ture has shown, the edges and the peripheries of culture are the places of 
more active semiotic processes, for the boundary “is the area of accelerated 
semiotic processes, which always flow more actively on the periphery of 
cultural environments, seeking to affix them to the core structures, with a 
view to displacing them” (Lotman 2005: 212). 

Let me further qualify such statements. From the perspective of the semi- 
otics of culture, cultural systems are dynamic systems that must be under- 
stood from a twofold perspective: from the inner and the outer point of 
view. The inner point of view is the conception of culture from the point of 
view of the culture itself, that is, from within. The outer point of view is the 
one of a scientific metasystem that attempts to reconstruct or describe that 
culture from an outside perspective. From the inner point of view, culture 
is linked to its opposition, non-culture. This opposition is also conceived of 
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as information versus entropy, order/chaos, and culture/nature. Moreover, 
from the inner point of view, culture does not need its outer, chaotic agent 
but can be understood immanently. As opposed to the consideration of 
the inner view given earlier, from the outer perspective, culture, and non- 
culture appear as two interrelated spheres. From the outer point of view, 
culture, and non-culture mutually condition each other, and they need each 
other. This is an important qualification to keep in mind because the theme 
of the alien is often couched in a dichotomic fashion, constructing an ‘us’ 
versus ‘them’ narrative. 

The theme of the alien being thought of as a form of extraterrestrial 
life has a very enigmatic status. This is a very complex and controversial 
subject, almost without limit. Today, this subject resurfaces in many forms 
of public discourses that are quite distant from the other, from ‘alterna- 
tive’ and ‘truth media’ to mainstream media. To these media outlets, one 
should not disregard the personal accounts of those referred to as “alien 
experiencers” or “alien abductees”. Extraterrestrials are generally charac- 
terized by a physical appearance that makes them somewhat different or 
‘other’ than human beings. Indeed, aliens encapsulate the idea of absolute 
otherness. The problem of the existence and appearance of aliens is shaped 
according to a twofold status. On the one hand, most of the representa- 
tions in Western societies are mediated from works of science fiction and 
cinema that depict extraterrestrials as figures marked by a shape and a 
culture that sets them in opposition to the lore of the common person. 

Taking up the topos of the monster as a creature without form — shapeless 
creature — and the archetype of the enemy that comes from the outside — 
the unknown — aliens are often depicted as ugly and malevolent beings. The 
alien and the enemy are quintessential representations of chaos. The alien 
comes from the edges of culture. It challenges rational thinking, violates 
the customs of civil society, contrasts with normality, and is an ultimate 
threat to humanity. Thus, the construction of the enemy and the demoni- 
zation of the other are often welded to each other in the representation of 
the alien. This narrative is sometimes counterbalanced with another, less 
frequent, and chronologically more recent motif that depicts the extrater- 
restrial as a benign entity who intervenes to support humanity and often is 
enmeshed in an anthropomorphic dimension. 

The link between alien, otherness, and foreign to a given culture also 
finds a correlation in the semiotics of culture. Lotman and Uspenskij, for 
instance, provide both a social and mythological reading of this phenom- 
enon. For Lotman and Uspenskij, the starting point is the high degree of 
ambiguity that characterizes the concept of “alien” (“čužoj”), to which the 
term izgoy (“outcast”, those who live at the margins of culture) is inter- 
locked. The alien or foreigner comes from a different culture and dwells 
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in a society and a culture that are foreign to her/him. For this reason, the 
alien shows a certain ambiguity and has a twofold connotation, for she 
or he belongs to two domains, the outside and the inside, the ‘own’ and 
the ‘other’. Undoubtedly, the alien can be seen either as an enemy, toward 
whom the ‘own’ community shows feelings of hostility and defense or as 
the holder of a particular knowledge (as in the case of shamans and sorcer- 
ers), that is, someone to be feared and respected with reverence (Lotman 
and Uspenskij 1985: 165). 


8. Conclusions: the phenomenology of an alien 


Today, the archetype of the alien has seen a reemergence through the wide- 
spread use of digital media and the massive proliferation of ‘conspiracy 
theories’. Such matters are at the forefront of discussion, arousing interest 
among experts and ordinary people alike. In some of these theories, the 
existence of extraterrestrial entities is supported, and the present, past, and 
future role of alleged “alien races” in the evolution of human beings are 
discussed. These accounts pay attention to the outward characteristics of 
extraterrestrials and their physical appearance, including facial features, 
in order to identify them, sort them out, and distinguish one from the 
other as well as to set them apart from human beings. We could, therefore, 
speak of a phenomenology of the alien, which also includes a description 
of their outer appearance coupled with other characteristics: culture, lan- 
guage, telepathy, and other psychic abilities, and whether they are benign 
or malign. While phenomenology is traceable in contemporary public 
discourse, it is also at a crossroads between ufology, alternative history, 
anthropology of aliens, and personal accounts of “abduction experiences”. 

It is important to note that, in general terms, in such narratives, there are 
two energies, regarded as positive and negative, and this is taken as a rule 
of thumb that applies throughout the entire universe. Aliens, too, fall into 
one of these categories. Some have a positive agenda, others very positive, 
while others are very depicted as dangerous and negative. In these narra- 
tives, it can be found some explicit references to entities that are identifiably 
classified through their external characteristics and, specifically, through 
some facial features (shape of the face, shape of the body, and color of the 
eyes) and other characteristics of the body (height, color of the skin, the 
number of fingers). According to this premise, there are different types of 
aliens that have different physical appearances. Some illustrations of alien 
groups are the following: the “Reptilians”, the “Mantis” or “Mantid”, 
the “Pleiadians” (or Nordic aliens), the “Felines”, and the “Andromed- 
ans”. As for the common body structure into which aliens can be gener- 
ally categorized, we can recall the following: “humanoids”, “reptoids”, 
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“insectoids”, and “Greys”. Reptilians are probably the most common and 
well-known species. They have been popularized, so to speak, by the many 
works of David Icke (1999). “Mantis” or “Mantid” has a green body, very 
large oval-shaped eyes, and a very small nose. “Andromedans” are thought 
of as having certain characteristic features, such as “a tall, slim body frame; 
very long arms; almond-shaped eyes and skin tones ranging from pale to 
olive-tinted. They are also said to have pointed ears and high cheekbones 
with a narrow face shape”.? 

The archetypes of UFO or aliens, in their positive or negative connota- 
tions, have often been used in literature and science fiction films so much 
so that it became a consolidated genre. This genre has often portrayed the 
encounter between human civilization and extraterrestrials, so much so 
that it crystallized in a particular topos, namely the encounter with extra- 
terrestrial race and culture. One of the first works of literature that tackled 
the theme of contact with an extraterrestrial race is the well-known science 
fiction novel The War of the Worlds (1898) by the English author H. G. 
Wells. It is one of the earliest stories to detail a conflict between mankind 
and an extraterrestrial race. This novel thematizes the theme of the alien 
invasion from Mars, capitalizing on the fear of negative aliens coming from 
outside to subjugate the Earth and the human race. It is worth remember- 
ing that in 1938 this novel received a lot of attention from the mass media 
when a young actor, Orson Wells, based a radio show on this book. The 
effect on the audience was so dramatic and powerful that it triggered a 
massive panic in the listeners, who believed the alien invasion from Mars 
had really happened. 

Another seminal text that brings us back to the theme of the alien inva- 
sion is the science fiction book Who Goes There? (1938) by the father of 
American science fiction, John W. Campbell. The alien is depicted as an 
elusive creature, hard to catch, who is able to infiltrate the human mind 
and take it over. In this connection, the film by Don Siegel, Invasion of the 
Body Snatchers (1956), should also be mentioned. This film was based on 
a book by Jack Finney. Phenomenal is also the transposition of the novel 
by Campbell mentioned earlier, which was made by John Carpenter, The 
Thing (1982). The idea of a non-hostile alien who is willing to cooperate 
with men can be traced back to the 1950s. The idea of the positive, non- 
hostile alien was put forward thanks to the contributions of writers such as 
Clifford Simak, Jack Williamson, and Philip Dick. I should at least mention 
a film that has put forward the idea of a benign, positive creature com- 
ing from another dimension: Close Encounters of the Third Kind (1977), 
directed by Steven Spielberg and E.T. (1982). 

To have or to construct an enemy is important not only for building our 
own identity but also to have an obstacle against which to measure our 
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systems of values and beliefs. It is not important whether this enemy is real 
or nor (Eco 1993). What matters is the construction and the demonization 
of the enemy. Enemies are different from us, and they behave according to 
customs and lore that are not our own. The other, the person that is differ- 
ent from us, the stranger par excellence, is the foreigner. In the Roman bas- 
relief, the Barbarians were depicted as bearded and snub-nosed. Indeed, the 
term “Barbarian” derives from the fact that Barbarians spoke a different 
language than the Roman language and, as such, were considered outsid- 
ers and a person that is not able to talk. The enemy is usually depicted as 
ugly. There is a general cultural rule according to which the enemy must be 
ugly because what is good is identified with what is beautiful, and one of 
the main features of beauty has always been what in the Middle Ages was 
called integritas. This means that one must have everything that is required 
by the average representative of that particular species. Therefore, if you 
are a human being, you will be regarded as “ugly” if a limb or an eye is 
missing, if the skin color is different from the average, or if the stature is 
below the average. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 

2 In this connection, see the special issue “Cultures of the Face” (Gramigna and 
Leone 2021) published in Sign Systems Studies 49 (3-4). 

3 In this regard, see the abundant literature on face disfigurement (Rifkin et al. 

2018; Skinner and Cock 2018). 

My own translation from Italian. 

A list of MA and PhD dissertations devoted to the UFO subject is available 

at: www.cisu.org/wp-content/uploads/2017/01/UFOTheses_by_Paolo_Toselli 

updated_20170101a.pdf (last accessed 11 November 2019). 

6 The bibliography on this topic is vast. Many recent pieces of research have delved 
into the issue from different perspectives philosophical, journalistic, and psycho- 
logical. This shows that academics and scientists have begun to question why 
such theories persist even though they are improbable and unreasonable and, 
above all, how it is that we have reached such considerable proportions today. 
Among the most recent publications, and for those who wish to explore the topic 
further, I would like to point out the research by Aaronovitch (2009), Wheen 
(2004), Thompson (2013), Lewandowsky (2021), Lewandowsky and Cook 
(2020), Lewandowsky et al. (2017), Damiani (2004), Cohnitz (2017), Leone 
(2016), Butter and Knight (2020), McCrea (2004), Pipes (1996), Oreskes and 
Conway (2010), and Eco (1990, 2021). 

7 An image of a quintessential representation of a “Grey” is found on the Wiki- 
pedia page devoted to UFOs: https://it-wikipedia.org/wiki/UFO (last accessed 21 
December 2019). 

8 Regrettably, I was not able to retrieve and consult this study. 

9 http://starseedsigns.com/andromedan-starseed/ (last accessed 20 December 2022). 
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Close Encounters of the Third Kind (1977) [Film] Dir. Steven Spielberg. 
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The Thing (1982). [Film] Dir. John Carpenter. 


9 Automatic faces 


The transcendent visage of 
trans-humanity! 


Gianmarco Thierry Giuliana 


1. Faces and alterity 


In the twentieth century, the West has had two great philosophers who 
have extensively dealt with the issue of otherness, and both have based the 
problem of the relationship with the other on the human face: Sartre’s gaze 
and Levinas’ face (Madou 2007; Legros 2017). This recurrence of the face 
is by no means a coincidence, and its capacity to undermine the “self” is 
part of common sense and knowledge: just think of the figure of the twins 
used in both ancient epics (Castor and Pollux in Homer’s Iliad) and cinema 
(Lisa and Louise in Stanley Kubrick’s 1980 The Shining; see Surace 2022). 

This close relationship between the face and the perception of identity- 
alterity explains why the face is now an object of study at the center of 
the most advanced research on major social and philosophical issues. 
Think, for example, of the collaboration between the humanities and com- 
puter sciences to address the problem of identity recognition in artificial 
intelligence-based face-reading techniques and technologies (Voto 2021). 
Advanced academic research again goes hand in hand with a widespread 
critical social consciousness about the dangers of certain types of facial 
representation and reading: from the controversy over TikTok’s “beauty 
filters” in our newspapers? to books on veganism that problematize how 
the removal of the face from the food we buy is a way of concealing the 
living being from which it came (Masson 2010). What’s more, with the 
advent of the 4E turn (Newen et al. 2018) in mind sciences, the works 
of these philosophers and their philosophical debate on the “face-other” 
relationship have been continued and recognized as a critical one in our 
century in a more interdisciplinary context (Pawlett-Jackson 2022). 

In this sense, the face is a philosophical, historical, semiotic, sociological, 
psychoanalytic, anthropological, and most of all an ethical object of study. 

Building on this premise, this chapter seeks to address the contemporary 
tendency of extending the category of “human” and “person” to “nonhu- 
man” beings and things by focusing on the role of the digitalization of 
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faces in such a process. Indeed, over the past decade, we have witnessed a 
fundamental rethinking and extension of the category of “human” within 
the semiosphere (Lotman 2005) of most cultures. On the one hand, this has 
occurred through the critique of anthropocentrism in relation to the ethics 
of the human-nature relationship (with the emblematic cases of sustainable 
development and vegetarianism as a cruelty-free lifestyle). On the other 
hand, we have become increasingly accustomed to qualitative interactions 
with anthropomorphic nonhuman agents (from Metahumans to Chat- 
GPT), and we are continuously exposed to narratives in which androids 
and robots are increasingly difficult to classify as nonhumans. 

This last aspect is the one we will focus on in this work and with a par- 
ticular attention to digital games which, from the point of view of both 
interactions with AI (Giuliana 2023b L’intelligenza) and of their stories 
(Giuliana 2021a L’oggettuale), are perhaps the most iconic example of 
such conceptual reorganization. The reflection on the contemporary rese- 
mantization of the human in the light of digital creation and manipula- 
tion of faces will be done in this chapter through the case study of the 
video game NieR: Automata (Square Enix 2017) and by devoting special 
attention to the comparison between the faces and visages of its androids, 
robots, and drones. 

To do so, we will begin with a brief exposition of the general plot of 
NieR: Automata in the first section, summarizing the main events and high- 
lighting story elements that challenge the distinction between humans and 
machines. In the second section we will analyze separately the faces of the 
game’s three protagonist figures: androids, biomachines (robots), and pods 
(drones). We will thus see how these three different faces of the mechanical 
can take on a “personality” and become expressions of humanity in three 
different ways: through resemblance-identification, ambiguity, and projec- 
tion. This analysis will allow us, in the third and final part, to come to two 
conclusions. The first will concern the rhetorical use of the face to con- 
vey the values of transhumanist narratives and undermine some of the key 
twentieth-century cultural distinctions. The second will concern the role of 
digital technology and in particular of digital faces in rethinking the human 
and the subject in a transcendental, ergative, and supra-individual sense. 


2. The story of NieR: Automata 


NieR: Automata (N.A. from now on) is a 2017 video game belonging to 
the genre of A-RPG (action role-playing game) conceived and directed by 
renowned game designer Yoko Taro. It is an award-winning game full of 
explicit philosophical references (From Pascal to Simone de Beauvoir and 
Sartre via Kant and Kierkegaard), and whose narrative has spawned both 
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the production of player interpretation/explanation videos viewed by mil- 
lions of viewers? and recently a NieR: Automata Ver1.1 series (2023 A-1 
Pictures). The game is set in 11945 ap, and its story is about a stereotypi- 
cal alien invasion of planet Earth by means of destructive machines called 
“biomachines”. In order to survive this invasion, which occurred in 5012 
AD, we are told that humans have created androids that can fight for them 
and have taken refuge on the moon. This gives rise to a proxy war that 
appears to have no way out. The current war against the biomachines is in 
fact the fourteenth, and the attempt to rid the Earth of the biomachines has 
now lasted for nearly 7,000 years. 

At first, the player takes on the role of a special type of advanced bat- 
tle android known as “YoRHa units” and specifically of a female android 
identified as “2B”. She will then soon be accompanied by a second 
advanced “scanner” android in the guise of a male boy and identified as 
“9S”. It immediately appears from the dialogues that these androids are 
advanced not only in terms of their combat capabilities but also in their 
intellectual and emotional capacities. In fact, distinguishing them from a 
human agent on a linguistic and behavioral basis is basically impossible. 
Nevertheless, behaving too similarly to a human, such as expressing emo- 
tions, is forbidden for androids. This is primarily because humanity (and 
its key characteristics such as emotions) is dysfunctional for the purposes 
of their mission, namely war. Moreover, although they are incredibly simi- 
lar to their creators, the androids worship the humans on the moon as true 
“gods in the sky” and dare not compare themselves to them. After the first 
few hours of play, therefore, we can see that N.A.’s narrative is grounded 
in four identity oppositions: 


. Humans versus aliens 

. Humans versus machines 
. Humans versus androids 

. Androids versus machines 
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These oppositions appear from the very beginning when the protago- 
nists encounter other characters who also share this “identity conflict” 
with them. An iconic example is the android merchant with a broken leg 
who refuses to replace it with a prosthetic one because then his whole 
body would be made of parts other than the original ones, and this 
makes him fear losing his true self (a clear reference to the dilemma of 
the Ship of Theseus). However, the deeper significance of N.A., hence its 
interest as an object of study, lies precisely in the questioning of these 
oppositions. Indeed, during their adventures, 2B and 9S will soon dis- 
cover truths that are baffling to them. First among them is that the story 
of humans on the moon is a lie invented by the head of the “YoRHa” 
units: humanity actually died out many years ago. In addition to under- 
mining the meaning of the mission of the protagonists, who came to 
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Earth to fight for the future of humanity, this revelation effectively 
makes androids the most human figure in existence. This fact will push 
the androids to gradually abandon their self-imposed limitations, which 
will allow 2B and 9S to develop a deeply emotional relationship that 
can be classified as much in the category of “big sister — little brother” 
as in that of lovers. 

The second revelation concerns the rebellion of the biomachines, which 
have killed their own creators and are therefore no longer fighting for them 
but to survive. In this sense biomachines and androids find themselves in an 
incredibly similar situation: both without their creators, both fighting only 
for themselves, and both capable of changes and afterthoughts that dur- 
ing all these years have made them different from what they were initially. 
This similarity between androids and biomachines becomes also physical 
through the figures of Adam and Eve: two advanced biomachines that look 
like humans. Not only that, but during the first finale we will see 9S aban- 
don his android body and transfer all his consciousness, personality, and 
memory right into a biomachine in order to be able to survive a virus that 
would otherwise destroy him. 

The third revelation is that some biomachines disconnected from the 
central network, developing an independent form of intelligence as a result. 
To make sense of the world, hence, these biomachines used everything that 
human culture had produced and left on Earth. Thus, 2B and 9S witness 
villages of pacifist robots, philosopher robots, religious sects formed by 
biomachines, biomachines with customs and traditions of human com- 
munities, robots with a parental sense, robots developing amorous and 
aesthetic dilemmas, repentant and suicidal robots, and even an “orgy” 
of machines desperately trying to reproduce themselves by emulating the 
positions and movement of human sexuality. In short, even biomachines 
become human through culture and an encyclopaedia (Eco 1975) which 
is not only intersubjectively shared with humans (Paolucci 2010) but also 
produced as humans (Violi 2007). 

The fourth and final revelation concerns the battle drones, called “pods”, 
that accompany the “YoRHa” units. At the beginning of the game, pods 
are actually very similar to machines in the sense that they are weapons 
with minimal logical and linguistic capabilities. Just as with biomachines, 
however, the pods accompanying the protagonists develop a kind of con- 
sciousness, free will, and a consequent evolution of their language. In fact, 
the secret “positive” ending of the game will actually depend on this very 
transformation. The pods, together with the biomachines, thus emphasize 
the idea that even the “simplest” machines can develop complex thinking 
and even a will: two traits typical of the human species. 

For the purpose of this chapter we will not examine further the plot of 
the game, which is very complex as much in its structure (multiple end- 
ings, story divided among three characters, and so on) as in its many hap- 
penings and their symbolic readings. We merely highlight here how these 
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revelations make N.A. a cult of transhumanist narratives and philosophies 
(Ross 2020), with an obvious problematization of the definition of the 
human and with a substantial critique of the presumed necessary correla- 
tion between being human and having a certain biological origin and his- 
tory. In short, we see how in this kind of narrative the “human” category 
loses its oppositional nature and becomes a participative one (Paolucci 
2010). Additionally, other scholars before us have also highlighted how 
the narrative of N.A. is fundamentally rooted in a problem of belief and 
in an issue of revalorization by having the players “playing on behalf of 
nonhuman, soulless individuals like the ones they believed they were fight- 
ing against” (Idone Cassone and Thibault 2019: 83). The question for us 
here will therefore be what role the face plays in this identity crisis of the 
category of “human” and how it is used to give the quality of “person” to 
what we would today still consider to be of the order of the mechanical. 


3. Faces of NieR: Automata 


As we pointed out in the previous section, the faces at the center of N.A.’s 
narrative are mainly three: 


1. The faces of the androids 
2. The faces of the machines 
3. The faces of the pods 


Figure 9.1 Square Enix. 2017. NieR: Automata. Detail of 2B’s face during a 
cutscene. Copyright: Square Enix (Fair Use). 
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We will therefore now make a separate analysis of these three faces to 
understand how they are the rhetorical elements par excellence of this tran- 
shumanist narrative. 


3.1 The face of androids 


As we can see in Figure 9.1, the level of definition of the android face 
is absolutely comparable to that of a human person (in comparison to 
other human faces represented in video games, that is): we can observe 
the details such as the smoothness of the skin and thickness of the lips, 
the specific shape of the nose, and even a mole. Therefore, it is not 
necessary to dwell here on how this android face is depicted in such 
a way as to be perceived as a person’s face. More interesting, how- 
ever, is to see in what way at the beginning of the game the android’s 
“personhood” and individuality are denied, namely precisely through 
an obscuring of the entire upper part of the face (or the lower part in 
the case of “operator” androids on the moon). That of the denial of 
humanity through the denial of the face is in fact a well-known semiotic 
construction with very antique religious origins (Volli 2021). As events 
progress, however, the discovery of the truth about the world will also 
correspond with an unveiling of the face and with a stronger sense of 
the self allowing 2B and 9S to make personally significant choices in 
the narrative of N.A. 

This unveiling of the face will reinforce an idea of a person not only 
from the point of view of their physiognomic features but also from the 
point of view of a symbolic reading of the face that accompanies their 
words and behaviors. Indeed, this function of the face is absolutely funda- 
mental to the recognition of individual members of human communities 
from the perspective of the cultural values and patterns by which the face 
is connoted (Giuliana 2021b Funzioni; Santangelo 2021). The symbolic 
dimension of the face thus gives 2B and 9S a supra-individual and collec- 
tive dimension. 

But the unveiling of the face has not only a symbolic role but also a 
very pragmatic one. Indeed, it allows the viewer-player of N.A. to be 
“touched” by the facial expressions of the protagonists at the most signifi- 
cant moments. Not only that, but in the specific cases of 2B and 9S this 
face also becomes that of a highly erotic object with a real “cult” that has 
arisen around its figure of 2B as a “waifu” and 9S as a “yaoi” character. 
As evidence of this, it is very easy to find a large production of fan-made 
erotic and pornographic products that have sprung up around them. In 
other words, from a rhetorical-experiential point of view, the unveiling of 
the face transforms 2B and 9S into “persons” through their pathemic agen- 
tivity on the off-screen human subject: completely outside the narrative 
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there is actually a human who, through the face, empathically relates to 
a machinic object (2B’s 3D model) in a manner similar to how he usually 
behaves and relates with flesh-and-blood faces. Such questioning of the 
unicum of the human through questioning of the empathic unicum of the 
biological face is today the basis of the social meaningfulness of hyper- 
defined digital faces such as those of metahumans (Giuliana 2022b What) 
and is possible here thanks to the narrative of N.A. 

Last but not least, the faces of 2B and 9S are also the player’s mask 
that interprets the story and at the same time the prosthesis (Eco 1997) 
that allows the player to see and be watched. The key moment here is the 
opening scene in which 2B’s new body is “programmed” (by setting vari- 
ous parameters) by 9S,* who addresses the player in the second person and 
comments on his choices as if s/he were 2B. Among these settings are all the 
visual parameters, and thus in essence vision, which corresponds in every 
respect with the vision of the player who has no other way of seeing the 
world than through the eyes and face of his or her avatar. In the most recent 
semiotic theory, it has been demonstrated very well how these prosthetic 
devices have subjectivity effects (Paolucci 2020), and we can therefore 
speak of a real pragmatic hybridization between faces. In conclusion, we 
see that in N.A. the face plays a key role in the attribution of “humanity” 
through its ability to define the “person” through six different functions: 


. Definition 

. Unveiling 

. Symbolism 

. Affectivity 

. Mask 

. Hybridization/prosthesis 


RNa BWwWN eR 


Figure 9.2 Square Enix. 2017. NieR: Automata. Screenshot of 2B next to a bioma- 
chine. Copyright: Square Enix (Fair Use). 
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Figure 9.3 Square Enix. 2017. NieR: Automata. Screenshot of the machine “Pas- 
cal”. Copyright: Square Enix (Fair Use). 


3.2 The face of machines 


Unlike the case of androids, recognizing the face of biomachines as having 
a “personification” role is far less obvious. In fact, in their most common 
form, biomachines look like this: 

As it is evident, the only features that can be traced back to a human 
face are the roundness of the head and two eyes. With respect to these 
machines, the first interesting thing to note is how those with advanced 
language abilities, such as Engels* and Pascal, often have some additional 
features such as a kind of eyelid and a kind of mouth. 

Secondly, the face of the biomachines is humanized through culture in 
the form of makeup and clothing‘: tube hats, tribal masks, clown disguises, 
ties, suits, ribbons, and so on. 

Thirdly, being the anti-subjects of the game and having a strong agen- 
tive power over the player (they can determine the player’s defeat) the 
biomachines’ faces constantly objectify the player’s avatar. Every time a 
machine looks at the character (it matters little whether it is 2B, 9S, or the 
third playable character A2) to address or to attack him or her, by virtue of 
the prosthesis previously illustrated, the biomachines turn their gaze to the 
player. So much so that to win, the player must constantly try to read the 
biomachines’ faces. Returning to Sartre, it is precisely this objectify- 
ing capacity that characterized the face for the French philosopher and 
distinguished him strongly from Levinas (Watson 1986; Jopling 1993). 
Now, the main property of the inanimate (such as the mechanical) is 
typically to belong to what is objectified by humans in virtue of their 
agentive power over it and, specifically, through the human gaze upon 
it. So we see here how the mechanical comes out of the domain of the 
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object, which does not necessarily coincide with entering the sphere of 
the human but certainly admits it as a possibility in terms of semiotic 
implication. 

The last point concerns the heterogeneity and ambiguity of the faces of 
the biomachines, which have at least two occurrences of explicitly human 
faces: the siblings Adam and Eve looking exactly like the androids and the 
“Red Girls”, the ego of the Machine Network (AI at the head of the bio- 
machines), who takes the form of a white little girl with big blue eyes, long 
black hair, and a small nose and mouth with features typical of a child. An 
ambiguity reaches its climax when the face of 9S, an android who in that 
context represents what is most human in the world, becomes that of a 
biomachine. This occurs the first time when 9S takes control of a machine 
(by hacking into it) to help 2B trapped in a factory full of enemies. In that 
first occasion 2B almost kills 9S, unable to recognize his usual visage. But 
it also occurs a second time in the first ending of the game when, because 
of a virus, he has to abandon his body by transferring his whole conscious- 
ness and memory into one near enemy. At that point only a green light 
coming from the eyes at that point marks his “nature” and can distinguish 
him from all the other biomachines. However, we have actually already 
seen that the biomachine Pascal has also green eyes, so this “trait” cannot 
actually distinguish an android from a biomachine as a general rule. It is 
rather the tender caring gaze of 2B toward the face of that machine that 
makes it “human”. 

In conclusion, we see how the machinic face takes on the connota- 
tions of a human face through a series of ambiguities and potentialities. 
Recognizing the potential humanity of the machinic face is thus not 
a question of emulating facial expressions or of physical resemblance, 
instead such acknowledgement depends on the undermining of a natural 
correlation (an interpretative habit) between a certain expression of the 
face and its hypothesized subjectivity as its content. The continuous rep- 
resentation of biomachines as culturally human constructs and the con- 
tinuous subject-to-subject relation with the faces of biomachines, during 
the tens of hours of the game and within its narrative, is a key part of 
N.A.’s message and meaning. This face-centered effect of humanity is 
then put into crisis toward the end of the game through the deliberately 
uncanny faces of Eve and of the AI that symbolically represents their 
increasingly hostile and “crazy” (therefore nonhuman) behavior. In the 
first case the “Red Girls” assume a very disturbing smiling facial expres- 
sion during a final cutscene, while in the second case the natural face 
of the humanoid biomachine assumes red eyes and has its skin being 
tainted in black patterns that redefine the original visage and conse- 
quently the identity. 
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Figure 9.4 Square Enix. 2017. Nier: Automata. Screenshot of a white pod hovering 
next to 2B. Copyright: Square Enix (Fair Use). 


3.3 The face of pods 


Thus, we come to the last and perhaps most problematic case: the flying 
drones known as “Pods”. Is it possible to say that these objects have a face? 
At first glance, it would be counterintuitive to say so. Indeed, they present 
four completely flat surfaces without any characteristic element of the face 
and with almost only arms to distinguish a “front” from a “back”. 

We have seen, however, that pods have a key role in the narration through 
a process of language and thought development that, by the end of the 
game, makes them characters on whom the happy ending depends. This 
“personality” opposed to their apparent objecthood then finds its coun- 
terpart in gameplay in two different aspects. Firstly, the player controls 
the pod separately from how he or she controls the androids. In this sense, 
although according to the lore these pods are controlled by the will of 
the androids, they are actually another machinic manifestation of human 
intentionality and agency. Secondly, there is an option in the game to “pet” 
or “high-five” one’s pod to “compliment” it, just as if it were our own 
domesticated animal. In these very moments there will be a real face-to- 
face interaction with the pod. A face-to-face was then re-enacted during 
some cutscenes of interactions between pods.’ 

Not only looked at and acted upon but also looking at and acting upon, 
here, within N.A.’s transhumanist narrative, it becomes extremely difficult 
to deny a face even to the pods. Both the player’s avatar and the player 
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project the face as a transcendental structure onto the pods’ top upper 
surface, associating their plastic features with their personhood (expressed 
through verbal language) and non-predetermined agency. 


4. Conclusions 


In light of this essay, we can draw two conclusions about three differ- 
ent themes touched upon in the different sections. The first conclusion 
concerns the rhetorical use of the face in transhumanist narratives as an 
expedient to represent the semantic extension of the category of human 
and the semantic hybridization between person and object. Such hybridi- 
zations of the face have already been observed and described by us in 
other video game and science fiction narratives such as in Cyberpunk 
2077 (Giuliana 2023a Faut-il). Above all, however, they are a real sty- 
listic trademark of the work of Yoko Taro, perhaps the most important 
transhumanist author in the video game field, who already presented 
such a view in the previous game (Square Enix 2010) outside any explicit 
reference to androids. In this prequel to N.A., the author imagines that 
humanity has developed a technology capable of temporarily separating 
people’s souls and consciousness from their bodies in order to be able to 
survive an incurable disease that would lead to the end of the species. For 
this purpose, replicants, genetic and soulless copies of living humans, are 
created to be able to withstand the disease so that, when life on Earth 
will be possible again, the “souls” (called gestalts in this state) would be 
reunited after the end of the disease. 

Due to an accident in the transformation process, however, it happens 
that the replicants themselves develop a consciousness and personality that 
prevents the souls from returning to their original bodies. This results in a 
loss of consciousness of the souls/gestalts who become hostile toward the 
replicants, taking on an appearance of shadows and merging together to 
form monstrous beings: 

The protagonist of the story, however, is unaware of this: he believes 
himself to be a human and considers the “gestalts” to be monsters that, 
through the player’s actions, he kills mercilessly throughout the game. So 
much so that he and his companion Emil destroy an entire village of “shad- 
ows”, effectively committing genocide. Without going into further detail, 
we also see here how the theme of the transformation of the human is at the 
heart of the story. And here, too, the author constructs a series of agents 
with nonhuman faces who look at and recognize each other as people. 
There is the face of the replicant protagonist Nier, that of humans turned 
into gestalt, that of the human co-protagonist “Emil” who undergoes ter- 
rible experiments and becomes a head-shaped object, that of “Grimoire 
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Figure 9.5 Square Enix. 2021. Nier Replicant ver.1.22474487139 .... A mon- 
strous giant gestalt (Hook) fighting against the protagonists. Copyright: 
Square Enix (Fair Use). 


Figure 9.6 Square Enix. 2021. Nier Replicant ver.1.22474487139 .... In order 
from left to right: Emil, Weiss, and Nier in a cutscene. Copyright: 
Square Enix (Fair Use). 
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Youhg Shade 


Okay, ready? What's a ship? 


Figure 9.7 Square Enix. 2021. Nier Replicant ver.1.22474487139 ....A shadow 
relates face to face with a robot with rudimentary artificial intelligence. 
Copyright: Square Enix (Fair Use). 


Weiss” which is a human turned into a talking magic book, and even that 
of some rudimentary intelligent machines. All these faces stand out and 
resemble each other at the same time, constantly confronting each other, 
representing the transformativeness of humanity understood as a set of 
emotional and reflective capacities. 

The second, and more theoretical, conclusion concerns the relation 
between the meaningfulness of digital faces and the complex and hetero- 
geneous human process of semanticizing faces (Leone 2022b Volti). Nar- 
ratives such as those of N.A. make great use of the digital face understood 
both as an image (the face of 9S with whom we empathize as in a film) 
and as a mask (the face we assume in the narrative as we act as of the 
characters in that story) and finally as a face prosthesis (the face through 
which we look and are looked at). In all three cases, the digital technology 
behind the video game allows us to relate to and even recognize ourselves 
with a digital face that is something undoubtedly other than the biological 
face and yet we are able to feel it is ours and to some extent true. From a 
semiotic point of view, this phenomenon can be explained from two dif- 
ferent perspectives. The first concerns the cognitive dimension (Paolucci 
2021) of the face as habit, that is, as an open semiotic structure capable 
of multiple adaptations and translations, which in the case of the game 
finds full meaning in the ludonarrative dimension (Thibault 2020) given 
by the intersection of N.A. history and its interactive-procedural mechan- 
ics (Bogost 2007). The second concerns the sociocultural dimension of the 
face that is increasingly showing itself as a secularized object through its 
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manipulations and decompositions made possible by digital technology. 
Indeed, the digital era is also culturally and epistemologically shaping our 
“mindset” (Bankov 2022) in this sense. This is quite evident by looking 
at the works on the digital face produced by the semiotic community in 
recent years, most of which seem to be moving in this direction (Viola and 
Voto 2023). Both of these points lead to the conclusion that the face has a 
strongly transcendental and ergative nature, as exemplified with the attri- 
bution of a face to pods. If a face may not have eyes, mouth, nose, ears, or 
forehead, then what are all these elements? A narrative like N.A. shows us 
how the elements we usually believe to be characteristic of the face, and 
therefore determinative, are actually overdetermining. Rather, the essential 
features of the face would turn out to be transcendentally and semiotically 
functional in its pointing to a being looked at and looking at that does not 
lend itself to immanent decompositions. That is, the characteristic elements 
of the face do not indicate what is essential but a surplus of meaning (Volli 
2008), making it a perfect semiotic object. Biological generation and semi- 
otic construction, one might even say semio-narrative generativity, seem 
to have co-determined themselves without us realizing it. The immanent- 
differential marking of the face as human, according to which, for example, 
what does not have eyes or what has hair on its nose is not human, reveals 
behind its obviousness an axiological structure. On the contrary, there is 
a virtual face in common with humans, animals, intelligent machines, and 
even “things” (think of pareidolia). Indeed, this is how in NieR: Automata 
the three faces which are clearly distinct in the beginning become equiva- 
lent in the end (a different expression of the same content). The recogni- 
tion of this latent face in all the agencies of reality is socially problematic 
because it undermines the historical connotation of the human face as the 
one belonging to the subject of an object-world. Indeed, it undermines its 
duties and rights because it coincides with a collective and transcendental 
vision of a fully agentive subject (Volli 2013). Reconnecting with what we 
were saying in our introduction, this is what is happening with animals 
(Leone 2021b On Muzzles) with whom we have always been face to face 
but whose visage we only recently recognize. That is why in cultural and 
communicative artefacts we will always continue to have faces appearing 
on what, apparently, has none in order to bring out a certain political 
idea of humanity. As scholars of human thought and culture, our task is, 
perhaps, not simply to highlight all the cases in which this phenomenon 
of resemantization occurs but also to suggest that this “addition” of faces 
(as trivial as a child drawing two eyes on the Sun) is not necessarily and 
not a priori ideological in a negative sense of distortion of reality (a force- 
ful attribution of a new content to an existing expression). On the con- 
trary, the emergence of visages on nonhuman agents may also be a way 
both to make us critically reflect on our own identity and to recognize an 
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intersubjectivity that is already there but is hidden behind that we fail to 
grasp and recognize not because of what it looks like but because of how 
we look at it from a given situation. It is indeed only through pragmat- 
ics and re-situation, two key characteristics and the main value of games, 
that N.A.’s post-humanist narrative assumes its full meaningfulness via 
the player’s gaze recognizing the visible transcendence (Pertseva 2017) of 
humanity in the faces of androids, machines, and pods. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
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2www.theguardian.com/lifeandstyle/2022/jan/02/ 
is-that-really-me-the-ugly-truth-about-beauty-filters 

3 www.youtube.com/watch?v=63PzQIbTrM8 

4 www.youtube.com/watch?v=b6WjJdwPTXQ 

5 https://nier.fandom.com/wiki/Machine_Examination_2?files=Machine_Examina- 
tion_2.png 

6 https://nier.fandom.com/wiki/Category:Amusement_Park_Machines 

7 https://nier.fandom.com/wiki/Red_Girl#Gallery 

8 https://nier.fandom.com/wiki/Pod_042 


Video games cited 


Nier (Square Enix 2010) [Video game] Dir. Yoko Taro. 

NieR: Automata (Square Enix 2017) [Video game] Dir. Yoko Taro. 
NieR: Replicant (Square Enix 2021) [Video game] Dir. Yoko Taro. 
Cyberpunk (CD Projekt RED 2020) [Video game] Dir. Adam Badowski 


10 Algorithmic faces 


Reflections on the visage in 


artistic translation and transition! 


Silvia Barbotto 


1. Introduction 


On the one hand, the dimensionality of contemporary living lies in the 
semanticized expectation of the physical, ontological, and phenomenal 
entity and, on the other, in its digitized and realized synthesis in the ethereal 
zone of e-society. The substantial polarities between the tangible and the 
intangible, between the phenomenal and the epistemic, matter and energy, 
physical and algorithmic, between the extremities of different events, are 
part of the same interconnected fabric. We think of categorical fractioning, 
typological definitions, and the intent of necessary objectification as parts 
of a functional but provisional phase: the imbalances caused by isolation 
with respect to a broader scenario need referential extension, integration, 
and completeness in order to fully realize their ephemeral independence. 

Among the fractional inferences there is the alternative of staying: in 
between is manifested the sense of a hybrid, pulsating, creative, extended 
world. In this space we find the nucleus of this chapter, whose basic pro- 
posal is to problematize the face as a solid entity in its own right in order 
to enhance instead its shaping context, identifying a field of research linked 
to an environment as well as an object. Without excluding the face as a set 
of elements with a compact and well-defined architecture, we position our- 
selves in that landscape that delimits its dialogical dimension, the scenery 
of senses, and effects but also that of data and clouds. 

Our objective consists in defining the points of contact and estrangement 
with respect to which the relationship of the encounter between the face 
and the situation in which it is immersed is generated; at the same time, we 
also have to probe what this situation generates, in terms of expressiveness, 
agency, and relationality. Hence, it is necessary to unravel the following 
issues, the resolution of which will be imbricated in the text: 


e identify the manifesting media of the human face and describe its expres- 
sive formants 
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e recognize the circumstantial dimensions and outline their compositional 
syntax 

e unravel the relational configurations of the face in contemporary society 
through a process of categorical association. 


In this way, we bring to light the vehicles of signification by which the 
meaning of the face is generated and shared in this intermediate space, thus 
procuring new perspectives of structured procedure, that is, new investiga- 
tive methods.’ Semiotics, as a discipline specialized in the study of signs 
and in the stabilization of fluid categories, is invoked to deal with the rela- 
tionship and transition between the magnitudes of binarism, fundamen- 
tally composing the mediation in which the processes of reading, writing, 
translation, and transition between one sphere and the opposite one take 
place: signification thus sediments in the incorporation and unbundling of 
the spaces traversed, of physical-algorithmic magnitudes articulated indi- 
vidually and collectively in the interlude that such dynamism generates and 
propitiates. 


Binarism and asymmetry are the laws binding on any real semiotic 
system. Binarism, however, must be understood as a principle which 
is realized in plurality since every newly-formed language is in its turn 
subdivided on a binary principle. Every living culture has a “built-in” 
mechanism for multiplying its languages. . . . For instance, we are con- 
stantly witnessing a quantitative increase in the languages of art. 
(Lotman 1990: 124) 


Interdisciplinarity in research leads us to hybridity as one of the theo- 
retical models: is the vacuum part of this place of transition? According to 
the Dirac sea model, the vacuum was an infinite space of negative-energy 
particles (electrons) which, by making a quantum leap, transformed them- 
selves into positive energy and thus left a gap, which was later found to 
be filled by the antiparticle, a virtual particle called the positron. The faces 
in the void bring us back to the investigation of occupied space as well 
as the material entity we refer to: virtuality becomes actualization after a 
quantum leap into the in-between. The leap between binarisms, the dia- 
logue in the living dichotomy, and the crossing into the void are bridged 
by everyday living in which the multiple languages of art proliferate and 
create the field. 

This space needs to “summarize the properties of quantum happenings”, 
as Merrell would say (2010: 83): paraphrasing his words, we must remem- 
ber that quantum particle-happening, as a possible possibility, can be in 
many places; moreover, we know that it does not live in an ordinary space- 
time “until it interdependently interrelates (and interacts) with something/ 
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somebody else”. The chronotope of the face within the paradigm of quan- 
tum physics is never the same: “you can never step into the same semiosic 
stream twice” (Merrell 2010: 82). 

Attributing to the body, and especially to the face, the predominant 
space in which to study these transitions, we see that since ancient times 
the human being has conciliated the interactions between aesthetics and 
materiality as integrated proposals. Inasmuch as they can be considered 
natural platforms, faces can be thought of as semiotic apparatuses that 
lend themselves not only to constant reading but also to the continuous 
intervention, direct or mediated, of human doing in relation to human and 
nonhuman doing and being: they are therefore predisposed to becoming 
cultural, social, artificial, and somehow algorithmical from the day they 
are born, in both proprioceptive and exteroceptive ways, becoming possi- 
ble possibilities. To say nothing of the nonhuman faces, specifically the face 
products of machine agency, which are certainly and completely artificial 
and which give rise to a whole series of contemporary questions to which 
it is urgent to find answers. Taking up the etymological sense of the fun- 
damental component of artificiality, we see that its roots in human action 
are also related to the paradigm of art as “ars facere”: we thus refer to the 
scenarios of a complex landscape, to the contemporary artistic condition of 
highlighting certain trends, especially in terms of intelligence and creativ- 
ity, both human and artificial. 

Some of these issues, void-polarities-artificialities, have already been 
addressed in a few open-access articles within the FACETS research pro- 
ject: the Socio-cultural Habits and Inflections of Faces Across Time and 
Space (SHIFT) program contains titles such as “Artificial Face and Tran- 
shumanism in Contemporary Art”, “Art, Face and Breathscape: From Air 
to Cultural Texts”, and “Trace and Traceability in/of the Face. A Semiotic 
Reading Through Art” regarding the topic of art/artifice, while the Visage 
Intelligence Systems From Antiquity to the Genesis of E-Societies (VISAGE) 
program includes, for instance, “Reading and Writing in N-dimensional 
Face Space”, concerning the intrinsic relationship between facescape, emp- 
tiness, and the neutral mask, with reference to Jacques Lecoq and the neu- 
trality of expression as a result of the search for the in-between space. 

The syntagmatic configuration of faces constitutes an environment that 
we could call the facescape: while the embedded face tends to give evidence 
of the material substance whose form it conveys, the virtual face, on the 
other hand, tends to become the spokesperson for a transitory demateriali- 
zation destined initially to occupy the ephemeral state of binary language 
and then, at times, to slide into that of disappearance. 


All figurative syntax is based on the interaction between matter and 
energy, and the stable or unstable balances of this interaction produce 
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identifiable figures. From the moment objects are treated as interacting 
bodies, and not only as abstract forms, further integrated into a figu- 
rative syntax, the interaction between matter and energy takes on the 
aspect of an interaction between the movements of one and the sheath- 
ing of the other. 

(Fontanille 2008: 226; our translation)’ 


While the digital images of faces perpetuate, proliferate, and, fleetingly, 
are sometimes illusorily forgotten, the data linked to them instead establish 
real material mechanisms. The physical-algorithmic extension therefore 
interests us from various points of view concerning above all its heteroge- 
neous and multiform entity: we would like to understand what gradients, 
intensities, and varieties of physical-algorithmic bodies we know; how 
they are generated and developed; what types of corporeity they include 
or exclude and, above all, in what spaces such syncretisms take place, how 
they are grafted into the real and digital territory, and how their habit- 
ability is constituted. The content is structured in the following subchapter: 


Art and artificiality, the physical-digital body-face; 

The semiotics of living; 

Datafication and matter in the contemporary ecosystem; 
Dwelling in-between. 


2. Art and artificiality, the physical-digital body-face 


The human being, a kind of arkhano* exposed to artificiality and, there- 
fore, also to making art dialogues and interacting with materials of various 
kinds, from the heaviest to the most ethereal, from the most evident to those 
that are invisible, in an approach of modalizing like that of the alchemist. 


The alchemist was called the “creator”, this science was commonly 
called the “Great Art” and the alchemical process, the opus, took the 
name of “Great Work”... . Just like the opus, the process of art finds 
simultaneously its own outcome and its own configuration on several 
levels: the physical one of the materials treated, the visual and harmonic 
one of the forms, and the ideal one of the spirit, and finally the “sapien- 
tial” one of the complex and numerous meanings, also brought back to 
the perfect iconological unity of allegory. 

(Calvesi 1986: 10-19)* 


“Formativity”, a term dear to Umberto Eco [1970 (1968): 13-16], could 
integrate the notion of artificiality by emphasizing the material dimension 
and in some way the physical one in a sort of isotopy identifying in the 
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organism the synonym of form and in the producing (more than in the 
expressing) the forming action. The performative dimension linked to 
agency in either human or non-human typology as well as a modeling and 
transforming function are inherent to the sphere of artificiality. Referring 
to Pareyson, Eco recalls: 


In the work, the person forms “his concrete experience, his interior life, 
his inimitable spirituality, his personal reactions in the historical envi- 
ronment in which he lives, his thoughts, his customs, feelings, ideals, 
beliefs, aspirations”. This does not mean that the artist simply narrates 
himself in the work; in it he exhibits himself, he shows himself as a 
mode. 

[Eco 1970 (1968): 14, our translation] 


Art, having undergone enormous fluctuations on every scale, as has soci- 
ety, is of its own disposition inclined to contribute to meaning, methodol- 
ogy, spatiality, and temporality in order to retrieve old narratives and, at 
the same time, to build new ones. Art, as a story-telling which identifies 
in saying also a form of doing, is not only a rhetorical way of reducing 
the speaking process of re-producing something preconstructed but also a 
manner of contributing to the production itself, as a continuous reference 
to the extra-societal setting. At the same time, recalling Deleuze’s view, in 
this posture there is not necessarily an intention to communicate; accord- 
ing to him, the relationship between art and communication is nonexistent: 


The work of art is not a tool of communication, it has nothing to do 
with communication, the work of art does not contain information, but 
on the contrary, there is a fundamental affinity between the work of art 
and the act of resistance (previously defined as the only plausible pos- 
sibility of counter-information). 

(Deleuze in “Art as an Act of Resistance” )6 


Visual anthropology, aesthetic ontology, biopolitics, and biosemiotics 
of culture and body are some of the disciplines which, without necessarily 
resorting to the paradigm of communication, deal with the artistic text, its 
production and proliferation, and the mechanisms that the text triggers 
directly or indirectly as a fundamental aspect capable of casting a bridge 
between tangible and possible societies. A dimension of potential, and 
therefore linked to the future, is often impregnated in making art whose 
objective, however, is usually far from the directive function of being a 
unilateral trajectory, coming closer instead to embodying indexical and 
certainly evocative suggestions. Jury Lotman, one of the major exponents 
of cultural semiotics, refers to art as a secondary modeling system, whose 
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dialectic focuses, as in the games paradigms, on the inventive dialogue 
between the primary language, certainly communicative, and multiple 
natural-cultural instances. 

Art, as a paradigm of reference on which to base oneself and from which 
to unfold, “is a singular composition of chaos through form, gesture and 
environments that take on a concrete corporeity in the space of communi- 
cation, vision and projection” (Berardi 2018: 29, our translation). As a pre- 
ponderant part of the implicit epistemic foundations, these are discursive 
ruptures of monolithic criteria whose distancing from the pre-established 
amplifies horizons within a liberation in the alternative. Art as an alter- 
native created language, evident avant-garde work, and constant method 
represents a process very similar to science as in Heisenberg’s affirmation, 
quoted by Formaggio: 


Both science and art give form over the centuries to a human language 
by means of which we can speak about the most remote parts of reality, 
and the coherent series of concepts such as the different styles of art are 
the different words or groups of words of this language. 

(Formaggio 1976: 80, our translation) 


Thinking of art as a semi-systemic world but also extending the inclu- 
sive perimeter, as an aestheticized panorama and a widespread means of 
evident artificiality, leads us to the necessary untangling of the spatial 
morphogenesis of digitality and analogy, which will be the second part 
of this chapter. The philosophical contribution of this text consists in 
conjugating the analysis of works of art with assumptions and deductions 
about how the echo generated from artistic awareness is pervasive, inso- 
far as it is a signification of the potential actualized through the recogni- 
tion of existing otherness, the fundamental crossroads of an immanent 
congruence, the combinatorial designation of sensitive variables, and 
co-possibilities. 

In 2001, Strauss and Fleischmann academically presented their artis- 
tic project Murmuring Fields, the prototype of an informative universe in 
which real space becomes a diffuse and hybrid environment, interfacing 
and interacting with the virtual, enabled by and with the body, defined by 
the researchers themselves as “a physical interaction space filled with data” 
(Strauss et al. 2001: 2). Mixed reality, they explain, 


signifies the interconnection of the real and virtual producing a new 
framework for communication and interaction possibilities . . . ampli- 
fied through the notion of a shared environment: a situation in which 
participants discover their interdependence in exploring, perceiving and 
creating the world. 

(Strauss et al. 2001: 3) 
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Already in Liquid Views’ — Narcissus Virtual Mirror (1992-93/2007), 
Strauss e Fleischmann had created a compelling avant-garde work, where 
the virtual touch played the role of an interface between digitality and cor- 
poreity, where sound and image interacted and blurred. It consisted of an 
artificial water platform in which, when used as a mirror, one’s own portrait 
was reflected as in a real surface: the action of looking was accompanied 
by the sound of water movement. The body, its contemporary action, and 
its inhabitation are to be contemplated in close connection with its variant 
and virtual, algorithmic integrations: the data and matter of which they 
are made are not merely a binary series of abstract numbers but are also 
founding elements of devices in which to mirror and construct one’s own 
identity in relation to that of others. This subjectivity is in co-construction 
with the changing world and its systematizing devices. 

We observe the human being in their trans-subjective and inhabiting 
extension in the environment, including both the tangible, terrestrial dimen- 
sion, and the digital dimension of cyber space, made up of data and infor- 
mation matter. During the Cuerpo, sujeto y reflexion semiotica® congress at 
the University of Granada (2022), there were numerous opportunities for 
reflecting on the cyborg dimension. In the round table El cuerpo Ciborg: 
del mito al sujeto posthumano,? presided over by Domingo Sanchez-Mesa 
and Nieves Rosendo, the process of cyborgization was considered as one 
of the possible ways of overcoming the dual gender, as well as approaching 
the establishment of a new, primarily political paradigm. 

Filippo Silvestri, too, in his contribution titled “Body and mind, their 
lives in extension. Phenomenologies and semiotics of the new post- 
pandemic cyborgs”! profiled how the new subjectivizing dynamics are 
necessarily and a priori cyborgs. Let us recall, for instance, artists such as 
Manuel de Aguas with his implant on both sides of the head designed to 
perceive and measure specific qualities of the environment such as atmos- 
pheric pressure and temperature, or the eyeborg Rob Spence, a filmmaker 
with a prosthetic eye consisting of a micro-camera inserted in the eyeball. 

“In the perspective of a semiotics of text and discourse — says Fontanille 
(2003: 3) - the body is above all a site of meaning, and of a meaning that 
takes shape from the sensations and impressions that this body experiences 
in contact with the world”. In experiencing the inhabited, the body situates 
itself by occupying a place that in turn welcomes it, envelops it, expels it, 
and that, in any case, assumes it as cohabitant: the human-environment 
relationship, according to this inclusive perspective, is bilateral. Technol- 
ogy has since ancient times interposed itself on this connection. 


3. The semiotics of living 


The semiotics of living, the matter of the world and nature, of the visible 
and the invisible, as well as bodies of all kinds and species, also change in 
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contact. It is a question of appraising the receptivity of the environment 
from which the human being intervenes and acts, plastically and figura- 
tively, in the interpenetration of a mutual synergy. Emerging and show- 
ing, subtending and concealing, protruding and welcoming, dichotomies of 
apparent extremes interpenetrate with individual and social tendencies in 
which these and other polarities mingle and alternate with further semiotic 
axes: those of narrative structures intrinsic to cultural determinisms, those 
of textual independence that treads unthinkable paths, those of gestures 
without a category because they are inchoative in poems where rhythm and 
harmony do not reflect norms but become confused exceptions. 

The face, besides being a physiological and bodily, poetic, and alchemi- 
cal punctum, is also the central core of today’s digital society: its agentive 
and expressive gradients in relation to ipseity and otherness, whether living 
or nonliving, human or nonhuman, must be considered. It deserves to be 
recognized for its narrative value and embedded variant. In analyzing the 
face within the environment that we have just described, it is necessary to 
consider the topological instance, identifying internal, external, and tan- 
gential face-world relational forms and thus elaborating the trichotomous 
arrangement of 


e the face in the face 
e the face in the world 
e the world in the face 


Considering instead the aspectual instance, there are practices and trends 
that occur, over time, between the body-face-real and the body-face- 
artificial by means of technologies and prostheses. A kind of measure- 
ment and in some cases alteration (on incremental or decremental scalar 
gradients) of sensorial, aesthetic, and communicative properties on a prag- 
matic basis therefore takes place: this will spill over into the typologiza- 
tion of macro-areas (which stabilize and categorize, obviously simplifying, 
the man-machine relationship) with particular attention to the intervening 
thresholds. Marcello Ienca spoke of this in proxemic terms: suggesting a 
human-technology (and face-artificial intelligence) relationship based on 
cooperation rather than competitiveness, he divided the navigation space 
into enhancement, transformation, and recombination. 


Whatever exists — says Lotman (1990: 132) —is subject to the limitations 
of real space and time. Human history is but a particular instance of this 
law. Human beings are immersed in real space, the space which nature 
gives them. Human consciousness forms its model of the world from 
such constants as the rotation of the earth (the movements of the sun 
across the horizon), the movements of the stars, and the natural cycle of 
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the seasons. No less important are the physical constants of the human 
body, which posit certain relationships with the outside world. 


The limitations to which Lotman refers imply the conscious demarcation 
of a circumscribed dimensionality, the basis for any analysis as a semi- 
otic configuration. The reference to the immersive grandeur of real space 
denotes the position of the bodies of living beings within this context and 
necessarily contemplates forms, rhythms, and densities whose limits are 
partly unknown: the real, physical, tangible, accessible space interpene- 
trates and integrates with the liminal, fictitious, mythical, virtual, unintel- 
ligible space. 

In the uncertainty of unpredictability, vector tensors are made up of 
forces and factors that influence the functional field which, together with 
the structural field made up of notions as well as geometries and mod- 
els, gives rise to the semiotic universe of fundamental interactions. The 
spacing of the face is not limited to the material circumscription of the 
human, physical face, but it is from there that the generation of a complex 
argumentative textuality arises, branches out, and proliferates, which is 
unsettling and involving, but distinguishable and examinable. If, on the 
one hand, we accept the constitutive naturalness of human physiology and 
the physical apparatus in which it exists, on the other hand, we know that 
the steps for the definition of the natural are cultural, social, and intersub- 
jective, and, in their multiplicity, an interpretative and generative process 
of meaning is involved, which actively participates in the process. 

Physical naturalness brings us back to discursive naturalness; the natu- 
ralness of data signs, even algorithmic ones, is a utopia: “the mythical algo- 
rithmic disintermediation, in practice, seems to be of variable geometry” 
(Airoldi and Gambetta 2018: 36). Naturalness is intimately linked to neu- 
trality, and its illusion and nonexistence from an algorithmic point of view 
are addressed in the text titled “On the myth of algorithmic neutrality” in 
The Lab’s Quarterly. What elements are contained or excluded in the face 
from a computational point of view so that algorithms can do their job, 
namely discern and process data? What factors enter the selective range 
allowing for definitions and dictionaries, historicization, and analysis in 
the process of discretization, and how do mathematics and geometry come 
into play by subjectivizing rather than objectivizing? What ethics apply to 
the new materiality of dataified signs where we sometimes refer to indexi- 
cal faces of flesh and bone and sometimes instead to embedded pixels? 

In this material transition, the body takes on a digital life implying set- 
tlement in an elusive shadow space, where transient movements allow for 
a spasmodic coming and going, where the concreteness of the real is only 
partially brought back into the digital presence, striated, layered, and dif- 
fused. Granting the face the meaning it occupies is not only an interpersonal 
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matter but also trans-personal, realized between tangible bodies and the 
concrete environment through new mediations, through their intangible 
subtle versions. The genealogy of spatial management and gestation in 
faces, and between faces and their dwellings, is relevant to understand- 
ing algorithmic mechanisms within a complex society and possibly coun- 
teracting affirmative pulsations by seemingly infinite progressivisms and 
quantitatively accumulating codified, meticulously structured knowledge. 
The face is an interface and has always mirrored itself in the devices at its 
disposal to recognize itself, to find the counterpart that corresponds to it: in 
the reverberation, spatiotemporal residues are created, which we attempt 
to probe by propitiating the critical and innovative arrangement of trans- 
versal materialities that cross its structural plane. 


4. Datafication and matter in the contemporary ecosystem 


Contemporary ecology is evidently hybrid, a compendium of a shared field 
between the analog and the digital, of matter that is body and data, embed- 
ded data, dataified body: it is a space that expands in the inhabited but is 
conformed in and with the inhabiting body. 


To consider the face as a device that tells transmedia stories implies not 
only allowing oneself to be guided by the inchoative stimuli dictated by 
its surface, but also propitiating access to the deep levels and stratifi- 
cations of form and content. Such storytelling is, therefore, identified, 
processed, clustered, archived. 

(Barbotto 2022: 140) 


The metaverse, a romanticized extrapolation of a digital universe, is one of 
the contemporary versions of the aforementioned mixed reality: transme- 
diality emphasizes the inclusion of the thing, the object, and the matter in 
the sphere of the knowable and not merely in the abstraction of the distant 
ephemeral. If we continue to discern and polarize dimensions such as body 
and mind, external and internal, real and digital, and subject and object, we 
drastically reduce the possibilities inherent in intermediate states and the 
contemplation of an embedded vision, that is, the subjectivation of a think- 
ing body that conforms synchronously in the process of materialization. 
The environment, the terrestrial setting, or the space we are used to 
referring to as our surroundings is foregrounded not as a passive social 
medium but as a systemic foundation in which human beings wander and 
establish themselves, often brutally imposing themselves on the established 
situation. This stabilization has brought about catastrophes and disfigure- 
ments, but it has also questioned itself as it has made way for welcome 
rather than domination, allowing the inhabited space to be the sustenance 
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with which to collaborate, rather than the pivot on which to erect illusions 
of infinitude. We are surrounded by data, but in turn we surround them: 
Massimo Leone speaks in these terms in relation to pixels: 


We are surrounded by pixels. And we surround them. As soon as we 
wake up, we look at our e-mails, web pages, social networks in our 
mobile phones; we interact with the liquid crystal displays (LCDs) of 
household appliances in the kitchen, in the bathroom, in the car; the 
screens of our computers “speak” to us through patrons of pixels; as 
we travel, giant panels in airports and train stations tell us the times of 
arrivals and departures, pixel by pixel. 

(Leone 2018a: 5) 


It is this inter-penetrating circumstance that interests us: how we intervene 
in the data, how the data intervene in us, and how to unveil the mechanisms 
of this media co-action even though it is often concealed, compressed, and 
obfuscated. 

Parikka’s geology of media (Crawford 2020: 32) comes close to the theo- 
retical proposition that media are a kind of extension of the earth, rather 
than of the human being. And this seems even more pronounced if we con- 
sider the relationship between media and artificial intelligence established 
in the COVID-19 pandemic and partially maintained to the present day: 


[W]e all know that AI has entered practically all domains of our lives 
and will continue to do so; the current crisis has only made the move 
to AI speedier. Universities all over the world collaborate with govern- 
ments and corporations to build stronger AI units to analyze big data, 
to operate machines, to augment reality and create 3D learning envi- 
ronments. Many of these new AI applications will help in providing 
solutions for the challenges of the (near) future, such as controlling dis- 
ease and the consequences of climate change. But all of this is not self- 
evident. In these new AI centers, there is sometimes an ethical committee 
that advises on the possible risks and benefits of specific applications. 
(Eugeni and Pister 2020: 92) 


In Chaosmosis, Guattari (1992: 15, our translation) responds negatively 
to the question of whether we should consider the semiotic production of 
mass media, information technology, telematics, and robotics as a sepa- 
rate entity from psychological subjectivity: “The consideration of these 
machinic dimensions of subjectivation leads us to insist, in our attempt to 
redefine it, on the heterogeneity of the components that make up the pro- 
duction of subjectivity.” In considering the category of the nonliving active 
and tensive, of the animated object and dynamic media, which assume 
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a performative and coactive role in the environment and experience, we 
introduce the conception of thingness as pertaining to the sphere of the liv- 
ing: to do so implies attributing a presential breath capable of unraveling 
vibration and movement in a kind of distributed transversal propulsion 
to any material form, even those generally considered inert. Does a stone 
vibrate? 

The data perform a process of depersonalization, which amalgamates 
unitarities to form clusters and sequences, anonymous, heterogeneous, and 
multidimensional meanings, interpretations that create gaps and which, 
almost denying subjectivity, declare and datify by negation. The data 
perform: in the exhibition The performing data Fleischmann and Strauss 
(2011: 49) show their interactive media artworks from 1980 to 2001. “The 
Data Performers are involved in space-time environments which we call 
enterable spaces of thoughts (begehbare Denkraume), within which the 
viewer becomes the participant of an interactive plot. Inspired by Aby War- 
burg’s neologisms such as ‘space of thought’ (Denkraum) or ‘psychological 
containers of energy’ (psychische Energiekonserven), we develop aesthetics 
of virtual spaces of knowledge and thought. 

Interactive data networks have enlivened everyday life: their interest lies 
in understanding mixed reality and capturing technological intermediation 
through media art to make users more active and reflective. 


In order to transform information into knowledge, people need to make 
choices, compare, evaluate, and interact with others. Instead of intel- 
lectual and technical automatization for the processing of information 
into alleged knowledge — as computer science does — media art combines 
automatism of the machine with an act of uncovering of its structures. 
Data Performers, data mapping and visualization are used in order to 
give a new structure to the already existing knowledge, and, thus, to 
rediscover it (reference to works Home of the Brain, Semantic Map, 
Media Flow)”. 

(Fleischmann and Strauss 2011: 53) 


The sphere of digital data seems to have expanded and gained ground in 
the space of meaning, penetrating the forms of everyday life, and revealing 
itself to be matter as well as concept and event: automation transcends the 
industrial and military sphere, and artificial intelligence reaches all con- 
temporary spheres. Data and the digital information society are increas- 
ing exponentially and magnifying their scale, so much so that they are 
considered by some to be the fifth element, a new manifestation of the 
mass-energy aggregate: according to physicist Vopson of the University 
of Portsmouth, “currently, we produce ~1021 digital bits of information 
annually on Earth. Assuming a 20% annual growth rate, we estimate that 
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after ~350 years from now, the number of bits produced will exceed the 
number of all atoms on Earth” (Vopson 2021)."! 

In his latest publication he proposed an experimental protocol to test the 
validity and consistency of the mass-energy-information triad by proving 
both the information conjecture, that is, the observability of information 
matter in the universe and the existence of information as the fifth state of 
matter in the universe. Building on the principle of Landauer, who as early 
as 1961 claimed that bits had a precise physical consistency and associ- 
ated energy, and combining Shannon’s information theory with the laws 
of thermodynamics, the author extends these insights to the principle of 
M/E/I equivalence, that is, matter/energy/information. 

Of course, to think that a piece of limestone has the same substance as an 
artefact made of cotton, a concrete structure, an electroacoustic transducer, 
a light sensor or a set of data and big data, and that in turn all these ele- 
ments are subjectivized to the extent that they play a coactive and proactive 
role, is a risky step as well as a simplistic attitude. Digitality, as an instance 
of modalizing, a medium of interactive processes, a resilient approach that 
adapts an original package of signs into a conventionally adequate new 
script, is certainly a mechanism of invention. Its history can be dated in 
fairly recent terms: a few dozen years have been enough to allow it to 
pervade our daily lives. In this discreteness there is an incessant cymatic 
translation, from the text to a new format of itself and again from the new 
text to the semiosphere of interpreters; at the same time digitalization has 
allowed transversality to prosper more easily, enabling a horizontal inclu- 
sion of the users: interpretants and producers become the foreground of 
the sphere of digitality englobed in one identity, the prosumer. The act of 
enunciation is increasingly digitalized in its production, reception, and, 
of course, during the transition where a meaningful field takes shape, with 
its own consistency characterized by flowing waves captured by structural 
mechanisms known, in general terms, as medias. It is of great importance 
to recapitulate the role of the media which, as the etymology of the word 
itself points out, consists in offering a platform of mediation that, it should 
be noted, is always filtered, politicized, and, in a single word, semiotized 
not merely by individuals but by large corporations. 

It is inevitable to contemplate our contemporary entangled field as a 
contorted but extremely organized (also) digital one, where a continuous 
encoding and recoding takes place, where authority is partially dissolved 
but where even the identity procedure (construction, narration, sharing) is 
somehow disintegrated, distorted by new characteristics in the process of 
definition and understanding. Its material constitution and the contact of 
the enveloped body-faces with their interaction devices, the energetic and 
therefore material conformation of the binary codes, their remains, their 
disappearances, the visual but also epidermical and synthetical impact of 
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a virtual body-face with a flesh body-face, the highly prolonged use of 
certain cervical muscles when using technological apparatuses essential to 
making digitality live, are, among others, some potential topics for new 
research. The digital domain, which now seems to be going hand in hand 
with the analog world, does not require separate or preferential treatment 
but instead becomes the warp of a weft with which it constitutes an inevi- 
table weaving of our contemporary and future times: the face is, ipso facto, 
an intense conglomerate of both. To consider space as an agent and digital 
data as coauthors of this agency, we need to understand their exponential 
growth, their functioning within digital citizenship, their massive energetic 
constitution, and their extractive nature. 


5. Dwelling in-between 


Each set of molecules has its own construct of natural frequencies depend- 
ing on their structure, materials, and delimiting peripheries. In music, for 
example, the normal modes, and therefore the natural frequencies that 
emerge from the vibration of instruments, are called harmonics. And this 
makes us notice how the normal and the natural seem to belong to a similar 
semiospheric circumscription, which interpenetrates the artificial aspect. 
The matter, therefore, the materials that come into contact with each other, 
propose a semiosis attractive to interpretation. The artificiality of the face 
and the interaction between matter and flesh, analogical or digital, tangible 
or intangible, lead us to think of material pluralism as a substantial need of 
the human being: Jacques Fontanille’s studies of the semiotics of the body 
have thoroughly examined this field, and by combining his suggestions 
with insights from the domain of art, a syntactically readable narrative 
landscape of the face can be constructed. The ways in which some artistic 
proposals have interacted with various degrees of artificiality in both a 
performative and a representative way are possible topics to be analyzed: 
portrait depictions, olfactory/sound essences, make-up, masks, prostheses, 
from the embodied face to the digitized face, from the creation of stochas- 
tic virtual identities to bots and cyborgs. And in part we will do so. How- 
ever, it is necessary to understand how the artificial face moves beyond 
the artistic sphere and is instead diffusely installed in everyday life until it 
reaches isomorphic pervasiveness in artificial intelligence, which stimulates 
us to ask ourselves the following questions: how do art and artificiality 
interact in the contemporary world and how did they do so in the past? 
How are artificial faces articulated? What kind of relationships is formed 
when faces and other materials come into contact? What type of materi- 
als are those? Is the digital world also endowed with material that dia- 
logues with the face? Is the virtual face also a material face? In this chapter 
we answer these questions using ethnographic, inferential-abductive, and 
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argumentative-narrative methodologies based on a selection of the numer- 
ous bibliographies/videographies existing in the semiotic and literary fields. 

To deny the spurious entity of digital experience and corporeality would 
procrastinate the necessary and indeed urgent reflection on the in-between, 
on the evolution and crystallization of this hybrid reality. Homi Bhabha 
speaks of inter-mediality (in-between) as an internal difference, a viven- 
tial edge, an interstitial intimacy: “It is an intimacy that questions binary 
divisions through which such spheres of social experience are often spa- 
tially opposed” (Bhabha 1994: 13). We would be dealing with an other 
space: “by exploring this Third Space, we may elude politics of polarity 
and emerge as the others of our selves” (Bhabha 1994: 39). 

Besides, the intermediate space, as the place of non-dual being, is at the 
heart of many philosophies and cultures across the centuries: in “Being. 
Approaches to non-duality”,’” Jean Klein (1989), a musicologist and doc- 
tor of the modern Advaita Vedanta school, brings together the writings of 
various authors who reflect on the consistency of non-duality and refer to 
consciousness as instantaneous and direct perception of the self and thus of 
the world, syncretism between knower and known object, between percep- 
tion and perceived thing. 

The simultaneity achieved in interstitial positioning is still mediated and 
semiotized, but performative competence and presential disposition contrib- 
ute to the immersive grasping of knowledge. Now, the point is not to seek 
consciousness in the alternate dimension of digitality, and especially AI, but 
rather to set up ethical philosophical mechanisms whereby consciousness as 
a subjectivizing entity manifests itself transversally, embedding itself in the 
matter. In Chapter 5 of his book AI and Consciousness (2015: 117-150), 
Murray asks whether the aspiration of AI to resemble human intelligence is 
also reflected in the possible tendency of AI to achieve a state of conscious- 
ness and brings the reflection back mainly to the plane of the body and the 
transhuman extension of life. Vitalists of matter seek to recover this forgot- 
ten union with the beginning of progress understood as intensive production 
and tend to recognize the stratification of living in matter, in its multiple 
and sometimes ineffable forms of life, predisposing the liberation from the 
anthropocentric posture and approaching a — almost animistic — transver- 
sality of living: “I think it is both possible and desirable to experiment with 
the idea of an impersonal agency integral to materiality as such, a vitality 
distinct from human or divine purposiveness” (Bennett 2010: 125). 


6. Conclusions 


The twenty-first century is characterized by the predominant presence of 
a new increase in digital space, which is also the gateway and recepta- 
cle of a new materiality: under the long-standing influence of a capitalist 
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system attracted by privatization, one can imagine how the races for the 
privatization of this space and this new materiality are conspicuous: the 
artistic ensemble IOCOSE spoke about this during the presentation titled 
“Lip-synched Stardust” at the Academy of Fine Arts and Design of the 
University of Ljubljana. The ethereality that is thus often attributed to AI 
omits, erroneously, both the soft and the hard" apparatus made of heavy 
materials and minerals, the founding elements of privatizing, sometimes 
authoritarian and violent geopolitics: the creation, movement, and archiv- 
ing of this corporeity would, according to Vopson (2021), constitute an 
informational catastrophe. Jane Bennett, opting for the intrinsic vitality of 
matter, asks: 


How would political responses to public problems change were we to 
take seriously the vitality of (nonhuman) bodies? By “vitality” I mean 
the capacity of things — edibles, commodities, storms, metals — not only 
to impede or block the will and designs of humans but also to act as 
quasi agents or forces with trajectories, propensities, or tendencies of 
their own. 

(Bennett 2010: 10) 


To understand their trajectories, we must understand their movements, 
and to do so, we have to take a step forward: we can no longer consider the 
data we consume and generate as continuities lost in the cosmos. Instead, 
we are dealing with practices whose existence is an attempt at quantifica- 
tion that renders the abstract categorizable, that creates object bodies with 
magnitude; we are handling collective agglomerations stored in enormous 
centers of virtualized and, in some ways, already actualized if still latent 
meaning. How does the process by which information bubbles are formed 
work? Algorithms are the linguistic processes that underpin the function- 
ing of artificial intelligence: they are computer developments dictated by 
human programming and based on a binary code whose intricate combina- 
tions give rise to complex problem-solving processes. Bits are the basis, bits 
are data, and following the logic just adopted, data are matter and energy. 

The innovation of recent years thus lies in thinking of the information 
system as part of the ecosystem as well, thereby recognizing the material 
experience of data. At the same time, a major change is taking place con- 
cerning the origin of the instructions dictated to machines, which can now 
act retroactively" and process information whose instructions are inter- 
nally self-generated: this cumulative process conveyed by recursive neural 
networks is endowed with memory and is the basis of deep learning. In 
acquiring the capacity for mnemonics, as well as for classification (as in the 
case of supervised learning) and pattern analysis (as in the case of unsuper- 
vised learning), the machine equips itself with predictive mastery and thus 
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co-constructs the arguments on which it is prompted: although on the one 
hand this constitutes great functional potential that we witness as a daily 
occurrence, on the other hand there are also numerous ambiguities and 
possible degenerations. The fact remains that we are increasingly turning 
to computational knowledge to construct models in which human inter- 
vention is then almost peripheral or, in any case, posthumous. 

Similarities and contrasts between intelligences based on human and 
artificial neural networks, characterized mainly by retroaction and autom- 
atism, are a topic for reflection. It is necessary for us to consider data as 
multilevel aggregations whose value, besides being numerical with binary 
composition (at the basis of any algorithm there is a postulate based on 
+1, -1) is also and above all a social and cultural value. In fact, in order to 
arrive at such a label, the data must take an elaborate route, which often 
involves new forms of extractivism. This phenomenon is the thematic focus 
of Aksioma’s “Tactics & practice: new extractivism” talk series!® in which 
researcher and artist Vladan Joler models and visualizes various elements 
of the new engines of extractivism, enumerating, explaining, and graphi- 
cally representing them. The narrative path he proposes, which is rather 
dysphoric and alarming, sees the gravitational force as the starting point 
from which to establish a relationship with the environment, which is then 
fueled by social forces marked by the “new colonization”: algorithms are 
the transversal skills that order and determine the prescriptive modaliza- 
tions based on the modal value of power (what can and cannot be done). 
Interfaces constitute the mediation between the visible and the invisible; 
data are extracted, studied, sold, and aggregated into new bodies inhabit- 
ing new territories: everything we create conforms an extracted and col- 
lected content into an information bubble of great potential but enslaving, 
made up of fractals and new waste currently ingested. 

This reflection therefore has to do with the openness of the world, with 
eco-systemic action and performativity, and with the disposition and dedi- 
cation of the body. The physical-algorithmic face is the result of this hybrid 
becoming, the encounter of the environment and human-non-human inter- 
action. In Other Possible Dwellings. Nomadological Architecture’ the 
researcher Laura Rodriguez affirms that, despite the conflict generated 
by immersion in this dualism of parallel societies (real and virtual), the 
technological subject is a participant in its own transformation and in the 
places it inhabits, marking a real architectural breach between the two 
worlds from the point of view of habitability. 


“The technological subject, also known as the neo-nomad, makes use 
of technology to transcend time and space through the use of bodily 
extensions or interfaces that extend the limits of the senses and con- 
nection and interaction with other spaces or subjects. He is a subject 
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for which the concept of habitability no longer means being physically 
in one place or another, insofar as the new human-computer relations 
developed in today’s society have transfigured its concept, giving rise to 
changes in the relations between the local and the global, between places 
and spaces. [. . .] Architectures must begin to be designed taking into 
account multiplicity, the invisible space of the immaterial and its flows 
of connection, the cyberspace that constitutes the virtual city. The need 
for an architecture of interfaces and nodes makes the development of a 
rhizomatic structure essential”. 

(Rodriguez 126, our translation) 


In the hybrid and contemporary environment, the participating bodies, 
living and nonliving, are constituted in otherness through the topology of 
the limit that is both extension and retraction, that has a diffuse perimeter 
because it is the semantic seat of both intrinsic facts and writings and of 
propulsive and receptive tendencies: that is, it is on the threshold of the 
limit, in the conflict that is generated in it, that hybridity develops. The 
thing, the object, the nonliving opens its boundaries to cover dimensions 
that were previously excluded and solitary in that virtuality, so present on 
the screens, but so distant as consistency and material existence: the ten- 
dency to have exiled the data that we create and that, at this point in the 
evolution of artificial intelligence, are self-created from the most innovative 
algorithmic formulas has generated a sort of social and environmental psy- 
chosis of a schizophrenic kind. The solidified boundary is now fading, inte- 
grating in both senses. Reflecting on the semantic depth of digital space, 
including through the study and visibility of material backgrounds as well 
as through the analysis of the underlying transpersonal and sociopolitical 
dynamics, implies a new awareness of common sense, a propitious gateway 
to the humanistic and artistic spheres. Contemporary bodies move between 
hybrid dialogic tensions whose decentralization from anthropocentrism is 
necessary to fertilize the integration and cooperation of physical-analog 
and digital-algorithmic experiences and to facilitate, therefore, a critical 
but cooperative habitability in the meaningful consciousness of new eco- 
systems in the making. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 

2 The etymology of the word “method” comes from the Greek language as a 
composition of etd (meta) and odd¢ (odos, path). 

3 Original text: “Toda sintaxis figurativa se apoya en la interacción entre materia 
y energía, y los equilibrios estables o inestables de dicha interacción produ- 
cen figuras identificables. Desde el momento en que los objetos son tratados 
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como cuerpos en interacción, y no solamente como formas abstractas, integra- 
dos además en una sintaxis figurativa, la interacción entre materia y energía 
adquiere el aspecto de una interacción entre movimientos de unos y envolturas 
de otros” Fontanille (2008: 226). 

Veiled by a kind of mystery intrinsic to its very nature, “arcane” also means 
occult and scenario of the playing field of making (Ars + Khano). 

Original text: “Ľalchimista era chiamato ‘artefice’, questa scienza era comune- 
mente definita la ‘Grande Arte’ e il processo alchemico, lopus, assumeva il 
nome di ‘Grande Opera’. . . . Proprio come lopus, il processo dell’arte trova 
simultaneamente il proprio esito e la propria configurazione a pit livelli: a 
quello fisico dei materiali trattati, a quello visivo e armonico delle forme, e 
quello ideale dello spirito, infine a quello ‘sapienziale’ dei complessi e numerosi 
significati, ricondotti anch’essi alla perfetta unità iconologica dell’allegoria.” 
Deleuze, El arte como acto de resistencia, videoconference available in web: 
www.youtube.com/watch?v=Cq_zP4LSyik. Last consult 5 of September 2023) 
https://youtu.be/bjq1 3wyjhA8 ( Last consult 5 of September 2023) 

Body, subject and semiotic reflection. https://congresos.ugr.es/aes2022/ 

The cyborg body: from myth to post-human subject. 

Original title: “Il corpo e la mente, le loro vite in estensione. Fenomenologie e 
semiotiche dei nuovi cyborg post-pandemici”. 

Possible integration for further study: “Each day on Earth we generate 500 mil- 
lion tweets, 294 billion emails, 4 million gigabytes of Facebook data, 65 bil- 
lion WhatsApp messages and 720,000 hours of new content added daily on 
YouTube. In 2018, the total amount of data created, captured, copied and 
consumed in the world was 33 zettabytes (ZB) — the equivalent of 33 tril- 
lion gigabytes. This grew to 59ZB in 2020 and is predicted to reach a mind- 
boggling 175ZB by 2025. One zettabyte is 8,000,000,000,000,000,000,000 
bits. To help visualize these numbers, let’s imagine that each bit is a £1 coin, 
which is around 3mm (0.1 inches) thick. One ZB made up of a stack of coins 
would be 2,550 lightyears” (Vopson (2021). 

Original title: “Essere. Accostamenti alla non dualita”. 
https://aksioma.org/lip-synched-stardust 

Please refer to the chapter titled “The Mineral Layer” in Atlas of AI. Power, 
Politics, and the Planetary Costs of Artificial Intelligence by Kate Crawford 
(2020) for more details. 

Retroaction is often used in physics, but it can apply to any dynamic system 
that is capable of actively considering the results produced by itself and thus 
possibly altering its future, even prompting systemic changes. 
https://aksioma.org/new.extractivism.exhibition 

Original title: “Otros habitares posibles. Arquitectura nomadolégica”. 


11 Dating faces 


The facial space of belonging in 
online (dating) communities! 


Elsa Soro 


1. Introduction 


Social networking services (SNS) have emerged since the early 2000s as the 
primary platforms for the exhibition and representation of digital faces. 
Facebook, commonly referred to as “the book of faces”, has significantly 
established a new visual paradigm for presenting, circulating, evaluating, and 
modifying faces in the digital realm. The rise of digital photography, coupled 
with editing filters and applications, has enabled the infinite replication and 
alteration of digital faces. Consequently, the role of selfies, especially with 
the introduction of forward-facing smartphone cameras, has been extensively 
explored at the intersection of visual studies, media studies, and semiotics. 
The act of posting self-portraits on social media, often associated with narcis- 
sism, has been reframed by drawing on Foucault’s concept of the culture of 
self and McLuhan’s (McLuhan, 1964). media theories. Self-portraiture and 
its distribution through social media have become catalysts for understanding 
the relationship between the self and physical space. Notably, the Selfiecity 
project led by Lev Manovich and his team at CUNY (Tifentale and Manovich, 
2015) has provided insights into this dynamic, such as producing rankings of 
the most smiling cities based on selfie visualizations. Social media platforms 
serve as vast archives of faces, subjected to various haptic operations like 
scrolling, exploring, swiping, saving, and deleting digital facial avatars. 

The expansion of such digital face space has been further accelerated by 
the global COVID-19 pandemic, which forced people into isolation and 
transformed social and private spaces of interaction. Traditional social 
gathering places were temporarily or permanently closed, leading to a 
surge in digital remediation. Screens became the medium through which 
static digital profile avatars or dynamic fragments of the body captured 
by cameras on platforms facilitated interactions during periods of global 
confinement. Notably, screenshots of video calls on platforms like Zoom 
have emerged as digital traces of facial interactions, forming new mosaic 
compositions that garner attention on social media. Such phenomena have 
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sparked researchers’ interest in exploring the aesthetics of the digital face 
as exposed by different social and digital communities. 

While social media like Facebook and Instagram have been extensively 
studied as sites for identity representation and new aesthetic regimes, 
online dating platforms have not received comparable attention. Although 
research has examined self-presentation in online dating environments 
(Fiore et al. 2008; Ranzini and Lutz 2017; Degen and Kleeberg-Niepage 
2023), impression management related to profile pictures (Gibbs et al. 
2006), and users’ motivations for using such platforms (Hobbs et al. 2016; 
Blake et al. 2022), a specific investigation into the facial space produced by 
dating sites could contribute to a broader understanding of visual digital 
culture transformations. In such a vein, this study aims to explore the pro- 
cesses of digital image storage, production, and manipulation within dating 
sites and applications. Specifically, it will examine dating site interfaces as 
spaces that facilitate the remediation and recreation of face-to-face encoun- 
ters. On the basis of such premises, the work aims at approaching a wider 
facialization of the users’ “life” operated by the ecosystem of commercial 
marketplace; in such regard the case of the most popular short-term rental 
platform, Airbnb, will be embraced under the notion of home facialization. 

This research builds upon the author’s previous examinations of online 
dating platforms (Soro 2021; Soro et al. 2021; Soro 2019) where she exam- 
ined how tourism discourse shapes different modes of self-presentation in 
online intimacy. This time, the focus is how the architecture of popular 
dating sites invites users to browse through streams of facial images in 
search of attractive mates, thus commodifying the digital face. The analysis 
does not concentrate on a specific sample of apps but aims to understand 
online intimacy as a community discourse shaped within the polyphonic 
landscape of various dating sites and platforms. Through different forms of 
self-exposition, platform interfaces generate distinct processes of “reading 
the face” (or looking beyond the face). In line with Umberto Eco’s con- 
cept of intentio operis, this research aims to uncover the visual patterns of 
meaning created by different platforms using the archive of users’ uploaded 
and stored digital faces. 

To achieve these objectives, this work will be structured as follows: The 
following section will provide an overview of algorithm-driven online plat- 
forms as “platforms of the face”, which involve processes of facialization 
beyond Facebook. Then, the example of Airbnb, which combines stylized 
face symbols with place symbols in its logo (Bélo), will be briefly discussed 
to illustrate the creation of a facial space. The popular short-term rental 
platform will be interpreted as a “Facebook” of homes and domestic envi- 
ronments displayed in photo albums and subject to a reputational system. 

The second part of the text will examine online dating discourse as a 
marketplace, incorporating the dual connotation of market and place. 
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The commercial nature of dating platforms as intermediaries facilitating 
asset exchanges will be explored, along with the necessity of an intermedi- 
ary space for such exchanges to occur. The core of the study will involve 
reinterpreting the dating site interface, a topic extensively addressed by 
semiotic approaches. By delving into the etymological connotation of 
“interface” as being between faces, this notion will be analyzed as a tem- 
poral waiting mechanism and a spatial distance machinery that facilitates 
the production of online intimacy. The dater’s narrative programs (semi- 
otic versions of user experience), ranging from selecting profile pictures to 
engaging in swipe-and-match mechanisms, will be scrutinized. Lastly the 
piece will critically examine the curious resemblance between the Elo (the 
alleged profile score algorithm used by the popular dating app Tinder to 
group daters based on desirability) and the Airbnb Bélo logo. 


2. Face-platforms: Airbnb and the facialization of home 


The face as a meaningful dispositive and its controversial relationship with 
the notion of individuality has been the object of inquiry from different 
disciplines, from psychology to art theory, from neurophysiology to physi- 
ognomy, throughout history. In the present, the advent of the algorithmic 
face recognition technologies has revamped reflection on the politics of 
face (Gates 2011; Edkins 2015), often nurtured upon the notion of facial- 
ity introduced by Deleuze and Guattari (1980a).? In such a vein, one of the 
main controversial issues regards the alleged objectivity of face recognition 
systems that according to a critical reading? of this technology responds to 
existing power and class structures in the new surveillance systems. 

At the same time, in the context of the platform society (Van Dijck et al. 
2018), marked by so-called platform capitalism (Srnicek 2016), algorithms 
have regulated subjectivity processes, performances, and representations 
through reputational mechanisms, rating, and data surveillance, and self- 
quantifying practices (Lupton 2015). In such a vein, platform studies have 
excavated the setting, the interface mechanisms, and the correspondent 
discourse, engaging with uncovering strategies of self-commodification 
(e.g., Fisher and Fuchs 2015; Fuchs 2014; Langley and Leyshon 2017). 
The boundaries that a platform provides and its algorithm-based function 
determine the user agency within it and its behavior models. Furthermore, 
besides the intrinsic nature of the assets at stake and their potential mon- 
etarization, platforms enable space for storage and circulation of images 
crowdsourced by the users. Arguably those images constitute a new series 
of digital archives consumed under different scopic regimes enabled by the 
different platforms. In such a vein, it is not coincidental that Facebook, both 
metaphorically and literally, acquired Instagram in 2012. Through this 
acquisition the company initially owned by Mark Zuckerberg centralizes 
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the bonds and the interpersonal relationships and interactions under a 
unique social media aesthetic that regulates a sort of facialization of the self 
by driving the transformation of the user life in a chapter of a facebook. 

Beyond the aforementioned social networks whose initial core model 
has been centered around the profiling of users, a process of “facializa- 
tion” arguably interests a wider range of digital platforms that are based 
on a totally different business model. In such a vein, Airbnb constitutes an 
illustrative example of the aforementioned process. The Bélo, the Airbnb 
logo introduced in 2014, is graphically a geometric combination of a face 
that stands for the people, a location icon to represent place, a heart for 
love, and then an A for Airbnb. According to Airbnb rhetoric, the com- 
munity of belonging, visually illustrated by the Bélo, is inhabited in the 
platform by the figure of the host who interacts with the guest, through 
their facial avatars, by showing them their most intimate spaces, the home. 
Airbnb marketing proposition can be interpreted as a facebook of rooms 
and domestic ambiances “nicely” exposed in the user profile photo albums 
and subjected toa reputational system. Under its narrative of authenticity and 
diversity — as an alternative to the standardization of hotels — the brand 
arguably fosters a sort of mirror effect among the actors, both human and 
nonhuman, who inhabit the platform. 

Such reflexive logic embedded in the platform interface induces a 
visual homologation aimed at provoking in the members of the com- 
munity a “generic global familiarity”: host and the guest can easily rec- 
ognize each other through the common aesthetics of their profiles and 
their assets. Not only the furniture and the style must be adequate to a 
certain comfortably exotic common culture but also the micronarratives 
epitomized by the platform fosters a sort of “cultural homologation”, as 
shown by existing research on the racial biases promoted by the platform 
(Edelman et al. 2017; Kakar et al. 2016, 2018; Leong and Belzer 2016). 
Findings from research done by two researchers at Harvard University 
(Edelman and Luca 2014) show a widespread segregator usage among 
Airbnb hosts who were found to discriminate against potential “Black- 
sounding” renters. 

Lastly, such mirror effects can be extended beyond the domestic walls to 
the neighborhood and the cities where the properties advertised through 
Airbnb are located. In the 1990s, the urban sociologist Sharon Zukin 
(1995) crafted the notion of “domestication by cappuccino” to refer to a 
scenario in which urban public space is designed to be consumed through 
events and entertainment and by so doing rendered a space of surveillance 
and control against any form of urban conflict and resistances. With the 
disruption of platform capitalism, since the first decade of the 2000s, under 
the guise of its alleged reaction to tourism standardization, the Airbnb 
model progressively occupied housing units in contemporary tourist cities 
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encouraging gentrification and segregation processes (Guttentag 2013; Ball 
et al. 2014). 

Within the fancy Airbnb aesthetics, people and spaces (at different scales, 
from houses to cities) “mirror” themselves within the border of a “trust- 
worthy community”. The motto of the face-to-face encounter with the 
locals, as epitomized by the creative tourism discourse, produces a comfort 
vision field that excludes undesirable landscapes, homes, and faces. Seem- 
ingly, the online dating discourse, embedded in the site’s visual figures, 
themes, and architecture, produces tensions between inclusion and exclu- 
sion logics. 

In the following section, the face-to-face intimacy remediation enabled 
by dating apps and sites will be explored. 


3. Phenomenology of a date 


Grammatically speaking “dating” is the present participle of the verb to 
date and, according to the Cambridge Dictionary, means “regularly spend- 
ing time with someone you have a romantic relationship with”. Such a 
non-complex definition highlights yet at least two important aspects of the 
dating phenomenology: the act of meeting with someone for a certain time 
for a not better specifically “romantic” purpose. According to the different 
adjectives attributed to dating (for instance, casual) “dating” semantically 
moves closer towards the relationship domain of which dating represents 
the weak and uncommitted version. Dating then, in turn, represents a more 
“romantic” version of other intimate practices such as hanging out and 
hooking up. 

Despite the phenomenology of the different practices included in the 
vast domain of intimacy across cultures, the majority of them entail an 
encounter between two or more actors in a given space. In turn, narra- 
tively speaking, the encounter implies a previous phase that consists in the 
“search”.* Such a quest, in turn, implies an intermediation agent, which 
logically precedes the supposed immediacy of the actual intimate encoun- 
ter. Throughout history and across cultures such an intermediation role has 
been discursively performed by different actors, from the family, passing 
through wedding agencies, to the Internet, and finally artificial intelligence. 

In such a vein, the different studies on modern and contemporary court- 
ship (Adair 1996; Hirsch and Wardlow 2006) have stressed the substitution 
of the familyhood intermediacy with other agencies and actors, both human 
and nonhuman, who are responsible for supervising mating processes. The 
commercial nature of such intermediacy has been increasing, given the pro- 
fessionalization of mating activities and the advent of wedding agencies. 
Love and sex as consumption activities and the role of the market in shap- 
ing love, sexuality, and intimacy within the neoliberal paradigm have been 
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at the center of several critical analyses (Duggan 2002; Illouz 2007). Such 
interpretations implicitly or explicitly evoke the Foucault (1976) theory 
on sexuality as the pivotal factor in the proliferation of mechanisms of 
discipline and normalization and the gender disciplining of sex frequently 
theorized by Judith Butler (2004). Furthermore, the advent of the Internet 
gave birth to “old” neologisms, such as cybersex (Blair 1998), paving the 
way for new online articulations of mediated intimacy till the appearance 
of such “new” neologisms as sexting. These increasing virtualization pro- 
cesses of sexuality have provoked different reactions among scholars, just 
to mention a few, from the enthusiastic Giddens (1992) who advocated for 
its decentralization, freed of reproductive needs, to the pessimistic theo- 
ries of Bauman’s liquid love (2003) that entails a dissolution of traditional 


bonds. 


4. The facial marketplace of dating 


Within this debate, the irruption of online dating practices stemming from 
Web Personals, the first online dating site invented by Andrew Conru in 
1994, has represented a milestone for pushing further the analogy between 
dating and marketplace. The emphasis on the commercial exchange of 
goods has nurtured the main strand of literature on digital intimacy. In 
such a vein, studies on online dating (Heino et al. 2010) explored the ways 
in which the marketplace themes and figures resonate with online dating 
practices and how the market influences how users assess themselves and 
the others. The same dating apps and sites reinforce such an interpretation, 
playing with commerce metaphors: in such a vein the French site Adopte- 
UnMec logo represents a woman pushing a shopping trolley that contains 
a man; the parody of Amazon Dating pretends to offer “Hot Single Near 
You” for selling within the same e-commerce Amazon frame. 

“Marketplace” is a composed noun that implies the presence of a spatial 
dimension for the commercial exchange of goods. In this regard, leisure 
studies in turn have put the emphasis on the “place” part of the expression 
by stressing the role of spatial dimension as pivotal in understanding the 
progressive commodification of intimacy within the digital sphere. By con- 
sidering sex as a core aspect of leisure (Hardwick 2008; Devall’s 1979) and 
accounting for its pleasure-seeking dimension, leisure studies have initially 
explored the intermediacy role played by spaces of leisure such as bars and 
nightclubs, labeled as sexy spaces (Caudwell and Browne 2011), and how 
leisure practices produce space and inform the construction of place and 
community (Johnson 2008). Within this view, the role of the Internet has 
been referred to mainly within the transformation of place production. 

In such a vein, since the Internet has become a common tool used to seek 
friends and romantic and sexual partners, concerns have been raised about 
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the desertification of “real” places traditionally devoted to hook-up culture 
in favor of cyberspace accessed from desktop machines. Authors, especially 
in gay and queer communities (Mcglotten 2014), report a sense of loss for 
the replacement of public spaces for sex that “afforded rare opportunities 
for interclass contact and communication conducted in a mode of good 
will” (Delany 1999) with the advent of the Internet. Then, the advent of 
location-aware apps and the transformation of online dating to mobile dat- 
ing marked a disconnection from a static computer and a return to alleg- 
edly physical spaces (de Souza e Silva 2013) to enable a new configuration 
of the intimacy marketplace. 

The marketing discourse of the different dating apps emphasizes the pos- 
sibility that a potential partner is nearby or has just crossed paths and con- 
sequently turns the current location of the user into a potential courtship 
or hooking-up space (accordingly to the scope of the usage). In opposition 
to the dating desk activity, the location-aware devices would enable an 
alleged reappropriation of physical space, augmented by the presence of 
potential digital partners. The marketplace can also be interpreted as the 
setting for the production and exposure of self-presentation. In that regard, 
academic literature on social media (Chambers 2013; Bullingham and Vas- 
concelos 2013) extensively analyzed the process of production and pres- 
entation of the self-online, evoking Goffman’s self-presentation theoretical 
framework: one of the main interrogations has been whether the Internet 
has reinvented the social norms that guide us in everyday life. Against such 
a background, as Kalinowski and Matei (2014) noticed, “online dating 
websites are qualitatively different from many other online settings because 
of the anticipated face-to-face interaction” (p. 7). Such an assertion sug- 
gests an important shift in the intermediation agency. In contrast to social 
networks and social media such as Facebook, Instagram Twitter, and Tik- 
Tok, in dating apps, the online “operations” are prompted to lead to a 
face-to-face encounter. 

In this sense, Tinder’s former slogan “Match. Chat. Date”, clearly indi- 
cated a narrative path from the online (“match” and “chat” phase) to the 
offline (“date” phase) which was supposed to be the venue of the final 
performance, whereas the online space was considered as the space for 
the competence construction. The expected passage from the marketplace 
(online) to the face-to-face (offline), sponsored by the dating app discourse, 
foresees the role of an interface. 


5. Interfaces 


Commercial researches on social media interfaces have been extensively 
analyzing the composition of the page to promote the engagement of 
users (user interface), while the notion of UX (user experience) refers to 
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the feeling a user gets while navigating the page. Recently the human- 
computer interaction (HCF) has also caught the attention of the qualita- 
tive approaches aimed at focusing on the interpretation process entailed 
by such interaction. In particular under a semiotics lens, the study of 
interface “should also focus on interface sense production system and 
interpretation processes” (Scolari 2009: 4). Scolari, in his interrogation 
of the meaning production process in reference to a blog, has singled out 
different levels of analysis (plastic, figurative, communicative, and meta- 
communicative) and stresses the importance of the identification of the 
author (“the designer”) “ ‘footprints,’ marks, and instructions (affor- 
dances) inside the interface”. 

Following the theoretical tradition of visual semiotics, Reyes-Garcia 
(2017) has studied interfaces as images by focusing on “the visual mean- 
ing behind the construction of any authoring and exploring environment” 
(2017: 11). With the complexification of the WIMP paradigm (Windows 
Icons Menus Pointers) and the introduction of different types of media, the 
interface would transform the user into a media producer who interacts 
with other users through enunciative acts. In the interactive space of a 
social networking site, the semiotics approach to the study of the interface 
should aim at examining the way in which interfaces organize, enhance, or 
constrain the interactions not only between human users and computer- 
based systems but also between the users themselves. 

Thanks to the pivotal role of its visual patterns, the platform interface 
can be considered a text provided by a semiotics agency that shapes subjec- 
tivity and affective bodily and more importantly facial experience. Relying 
on such literature, in this chapter the online dating interface(s) are referred 
to as a time and machine machinery for the production of online interac- 
tions, aimed at producing an intimate encounter. In order to do so, we refer 
back to the etymology of interface. The noun “interface”, composed of 
“inter-” + “face”, was recoined by McLuhan and defined as “place of inter- 
action between two systems” (1962). According to the Cambridge Diction- 
ary, it refers to a connection between two pieces of electronic equipment or 
between a person and a computer. 

The prefix “inter” by meaning “between” thus implies the presence of 
a space between (a third) two actors, two sides. Synchronically the space 
in between makes user interaction happen by connecting the two parts 
within the platform space; diachronically such a gap can be interpreted 
as a lapse of time to spend inter (digital) faces within the platform border 
while waiting to leave the digital space and perform the “actual” encounter 
outside the platform. Such a time-and-space gap stages the presence of an 
outside as the ultimate scope of the platform service. In the next section, 
such spatial and temporal betweenness, embedded in the platform, will be 
analyzed, following the narrative path marked by a dating app site. 
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6. The user narrative journey interfaces 


6.1. Choosing the best face 


By drawing on the Greimassian narrativity terms (1966, 1970, 1974) and 
interpreting within such a framework the user experience in “standard” 
online dating sites, the dater narrative programs experience starts with a 
competence construction phase that consists in the login operation, the 
identity verification, and, most importantly, the registration process with 
the uploading of photos and the filling in of question forms in order to 
enrich the self-presentation details and allegedly increase the probability 
of finding affinities among users. According to the most common dating 
sites’ instructions and the interface visual patterns, in order to complete the 
profile, it is inescapable to choose a face. In the case of some famous dating 
apps, such a request is visually enabled by an empty face silhouette to be 
filled in with a digital portrait. 

On the topic of pictures used for one’s dating profile the Internet con- 
tains lots of buzz: from a rough scanning of the first blog entries within the 
search “Picture for self-portrait dating app”, all of them agree on a point 
that can be summed up as follows: the profile picture should “really be 
like you”. The importance of authenticity in the commercial online dating 
discourse partly arises from the attempt of contrasting deceitful practices 
such as the catfishing, which in mere visual terms entails the stealing of 
someone’s portrait. Beyond the fake profiles, users can freely and origi- 
nally reinterpret the space devoted to self-portraits. Empirical observations 
from a digital netnography on Tinder (Soro 2019) showed that a significant 
number of profiles browsed post a variety of alternative images to the face, 
such as food, pets, famous characters, or other objects with which to iden- 
tify their identity. Such images are thus subjected to a process of facializa- 
tion by performing the same functions that a self-profile does. 


6.2. Swiping faces 


Following the narrative program of the dater, once the “best” face (or its 
surrogate) has been chosen and the profile operations have been completed, 
the user is able to actually kick off the matchmaking procedures. In such 
a vein, each popular dating app arguably presents its own “semiotics of 
passion” (Greimas and Fontanille 1991) with which the site manages the 
face-to-face remediation and the relative emotions involved. The interac- 
tions during the matchmaking are therefore regulated by specific aspectual- 
ity strategies that emphasize in turn either the duration of the path toward 
an allegedly offline face-to-face meeting or the terminative point where an 
interaction stops. Profile swiping, the key navigation gesture popularized 
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by the dating site Tinder and also present in other dating apps (Bumble, 
Hppnen, HER, and so on), represents the iterative action that determines 
the potential beginning or ending of an interaction between users. While a 
swipe right (according to Tinder function) and a consequent match open to 
an indefinite time of waiting before an allegedly face-to-face meeting (that 
can be either undermined or punctual or constantly delayed), the swipe left 
marks the termination point of a potential interaction and consequently 
the exclusion of a given user profile from the vision field of the swiper. 

In particular, according to Tinder grammar, the swiping gesture is embed- 
ded in an environment similar to playing cards. As a result of the swiping, 
some “cards” containing users’ faces are stored in the list of matches; oth- 
ers are excluded and (at least in theory) never seen again. The serendipity 
of running again into a face in the offline world is thus avoided by the 
cancellation of the less desirable profiles (according to a given dater vision) 
and thus reduces the database of the selected pictures to the crush-at-first- 
sight ones. Following the performative stage of the narrative program, after 
the match between two faces (or their surrogates) has occurred, the daters 
are allowed to text each other in a chat box or in certain cases exchange 
supplementary pictures or videos or enable video calls. It is noteworthy 
that the range of possibilities has recently been expanded by some dating 
companies in order to cope with social distancing rules. 


6.3. Out of the interface comfort zone 


It is common to encounter in the digital sphere (i.e., in social media and 
in dedicated blogs) users’ stories about the disappointment caused by first 
offline meetings after chatting and flirting online for a certain time. Surely 
this chapter is not focused on the dater’s behavior (intentio lectoris) or on 
the intentions of the dating site designers (intentio autoris). The focus is 
rather on the meaning effects enabled by the text and how the text archi- 
tecture shapes and induces certain usages and interpretations (intentio 
operis). However, regarding the aforementioned disillusionment effect it 
can be inferred that the platform and its interface tend to create a comfort 
zone provided with its own borders, its own rules, and its meaning-making 
processes. 

The practice of sexting represents, in such a vein, the magnification of 
online dating as a leisure activity per se. Etymologically a portmanteau of 
sex and text, the practice became popular during the COVID pandemic, 
as shown by the data provided by same dating apps that registered an 
increase in new subscribers. At the same time, some popular applications 
such as Bumble and Hinge provided users with new features for publicly 
sharing their COVID-related dating preferences, and not surprisingly, 
including indirect references to sexting among the possibilities. Beyond the 
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contingencies due to the pandemic that forces both offer and demand to 
design new creative practices in the field of intimacy, “sexting” embraces 
a complex semiotics. It semantically combines two incompossibles lexema, 
at least in their literal meaning, since the action of text necessarily requires 
a medium and, on the contrary, the sex denies such intermediacy. In such 
terms sexting represents the ultimate figure of facial leisure, the interaction 
interface. 

Furthermore, following the reasoning mentioned earlier, sexting would 
finalize the narrative program performed by the dater within the dating 
system interface. This way, such practice stresses the paradox of the user 
retention encrusted in the domain of online intimacy, a service allegedly 
purposed for letting go of their users to the offline world but, in doing so, 
at the same time, leading them to lose their customers. 


6.4. The gaze of the Elo 


Lastly, the interface is responsible for enabling a certain gaze upon the 
other by embedding a certain scopic regime that determines the visual aes- 
thetic components of the interactions. It is noteworthy that the agency of 
a platform system in influencing user behavior and his or her “way” of 
seeing rests on an algorithm. In the earlier years of Tinder, the Elo, in a 
surprising assonance with the aforementioned Airbnb logo Belo, has been 
known as an algorithm responsible for scoring and rating the daters on the 
basis of their desirability. According to speculations, since Tinder has not 
publicly disclosed the specific details of their algorithms, a dater profile 
attractiveness is based on how many people swiped right on him or her; 
the more right swipes that person had, the higher his or her assigned score 
went up. According to the alleged Elo logic, a dating system serves and 
matches users with similar scores, creating a sort of internal communities 
clustered by their higher or lower desirability. 

Just as the Belo watches over the Airbnb community of belonging com- 
posed of trusted hosts and guests, with similar interior design tastes and 
lifestyles, the Elo seemingly guards the daters, grouping them in comfort 
zone communities, each of these characterized by a similar level of facial 
desirability. Following such an interpretation, the facialization of the inti- 
macy as designed by the platform architecture structures of meaning pro- 
vokes a reflection effect among the dater faces: arguably they looked back 
at themselves as in a collective selfie. 


7. Conclusions 


In its attempt to contribute to the comprehension of the extension of the 
digital face space within the realm of digital intimacy, this chapter acknowl- 
edges several limitations. Firstly, it lacks user data, relying instead on 
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references to previous research conducted by the author that involved inter- 
views and netnography. The focus of the current study is on the meaning- 
making processes facilitated by the architectural design of dating sites 
rather than on user experiences. Secondly, the analysis does not concen- 
trate on a specific platform or a set of platforms in contrast to the prevalent 
literature on online dating that often examines a single app or site. Instead, 
the research considers online dating as a discourse shaped by the collective 
presence of various commercial sites. Thirdly, the piece does not differenti- 
ate between mobile and desktop platforms, despite the significance of the 
spatial dimension in semiotics. The study does not directly address the dif- 
ferences in usability between portable and fixed devices. 

Lastly, and perhaps most importantly, the article takes a broader 
approach in delimiting the scope of the study, aiming to construct a dis- 
course on the production of online communities rather than focusing solely 
on specific business models or services offered by individual platforms. 
While platforms like Airbnb and Tinder, mentioned in the piece, exhibit 
clear differences in terms of services offered, communication methods, 
user demographics, and more, a comparative analysis of different platform 
models would yield a better understanding of the strategies of inclusion 
and exclusion facilitated by these platforms. 

Future research should aim to address these limitations by incorporat- 
ing a systematic collection of data. Additionally, a comparative analysis 
of different platform models is necessary to comprehend how the digital 
face undergoes tension and re-symbolization within the online sphere. The 
concept of digital intimacy should be further explored as a field of negotia- 
tion between cultural and social narratives, encompassing not only court- 
ship and sexuality but also broader discussions about the body and gender. 
Ultimately, it will be crucial to examine digital intimacy as an observation 
point for the emergence of new biopolitical aesthetics shaped by practices 
such as artificial intelligence and machine learning. 


Notes 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 

2 Visagété [Faceness] is a section of Thousand Plateaus, where the French authors 
juxtapose the head and the face incompossible. 

3 Kelly Gates in Our Biometric Future claims that the pursuit of facial recognition 
technology, ruled by the priorities of law enforcement and state security agencies, 
has destructive social consequences. 

4 In the actantial model, within Greimas theory of narrativity (1966, 1970), the 
actant subject aspires to join to an object. 
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A semiotic investigation of digital 
memorials and commemorative 
practices! 


Federico Bellentani 


1. Introduction 


Memorialization is a fundamental and omnipresent human practice that 
is increasingly articulated through digital technologies. Digital memori- 
als and the application of digital technologies in commemorative practices 
are thus spreading very rapidly at both institutional and vernacular levels. 
Digital memorial here refers to the usage of digital technologies and online 
platforms for remembrance and mourning after the end of physical life 
of one or more individuals or to commemorate an event or a place. They 
can be institutional or vernacular, static or dynamic. They can be digital- 
native, that is designed exclusively with the use of digital technology or a 
transposition of a physical memorial into digital forms. Finally, there are 
memorials including both physical and digital elements. As physical ones, 
digital memorials are designed by one or more authors (hereafter design- 
ers). Besides their purpose of commemoration, digital memorials inevitably 
represent the designers’ intentions and narratives, who decide what is to be 
remembered and what is not (Tamm 2013). The contents of the memorial 
are thus subjected to the designers’ control and curation (Moncur and Kirk 
2014: 968). 

However, digital memorials are open to a myriad of interpretations 
depending on the points of view that the users take towards them. Despite 
design strategies being available to entice users along a specific interpre- 
tation, designers do not have complete control over the interpretations 
of digital memorials. Users thus interpret them following their opinions, 
beliefs and feelings. Today’s digital technologies and platforms stimulate 
the collaboration of users, being thus far from traditional authorial logic 
and constraints. Yet their promise of diversity and participation needs cri- 
tiques and explanations (Section 3.1). The human face is a central element 
in digital memorials as it was for the offline ones (pictures on tombstones, 
funeral keepsakes, public statues and so on). Many digital commemorative 
practices are also centered around the face: portrait pictures on electronic 
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memorials are often the inter-face through which users can access data 
of the dead (Section 4). Exploring the connection between memorializa- 
tion, face and digital technology experience, this chapter investigates dig- 
ital memorials and commemorative practices and the meanings of faces 
within them. To do so, it first develops a semiotic-oriented framework for 
the design of digital memorials to ensure that digital technology remains 
centered on human needs and meanings (Section 3). This framework helps 
to sensitize designers to focus on participatory engagement that they can 
exploit in the development of new digital memorial technologies and plat- 
forms. The chapter then goes on applying this framework by analyzing 
three kinds of digital memorials and commemorative practices, especially 
focusing on but not limited to those centered on the face: (i) institutional 
digital memorials (Section 4); (ii) open, creative and playful solutions to 
reimagine contested monuments and memorials (Section 5); (iii) institu- 
tional and vernacular digital-native commemorative practices (Section 6). 
Before this, the next section explores the increasing digital transformation 
of our life and outlines the main developments that have made semiotics 
hold a significant position in the study of digital media, communication 
and technology. 


2. Semiotic approaches to digital media, communication 
and technology 


Digital technology has brought about many changes in society and contin- 
ues to do so by increasingly integrating into more and more aspects of our 
life. The concept of digital transformation has been introduced in 2000 to 
refer to these changes (Patel and McCarthy 2000) and has become broadly 
used by both researchers and practitioners since the 2010s (e.g. Reis et al. 
2018). Digital transformation refers to the adoption of digital technology 
by organizations and businesses to improve value, efficiency and innova- 
tion, but it can be seen as a broader process impacting all aspects of life. 
Numerous definitions of digital transformation have been proposed, but 
there are two common features that are common to all: it is a continuous 
process, and it triggers changes and potential disruptions by combining 
“information, computing, communication, and connectivity technologies” 
(Vial 2019: 1). 

In today’s neoliberal global political economy, there is a positivist idea 
of digital transformation, a general tendency to consider it a hopeful evo- 
lution of human capabilities and abilities. According to the Anderson 
and Rainie (2018), digital life enriches family and community relations, 
improves work creativity, and provides health and wellness. In this view, 
digital technology is seen as a problem solver, an education tool, a travel 
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companion and a memory aid helping humans to reach a higher qual- 
ity of life. Beyond this, critics have focused on the dark sides of digital 
transformation, seen it as a process that is not sustainable nor equita- 
ble, creating new (digital) divides at the societal level (e.g. Barekat et al. 
2017; Horl et al. 2020). These critical studies provide theoretical and 
methodological frameworks to assist researchers, practitioners and poli- 
cymakers to shape a more human-centric digital transformation. Hence, 
new digital humanities approaches are appearing focusing on ethics and 
philosophy of technology, the relation between technology, humans and 
the arts, participation and democracy on digital platforms, the digitali- 
zation of tangible and intangible cultural heritage, and so on (Werthner 
et al. 2022). 

Still today there is a gap between enthusiasts and catastrophists of digi- 
tal transformation which is difficult to overcome. In his essay Apocalittici 
e Integrati (1964), Umberto Eco defined intellectuals through an oppos- 
ing category: as apocalyptic and opposed to mass culture or as integrated 
and dedicated in keeping with mass culture. This category has often been 
applied to define a general attitude towards digital transformation, new 
digital technologies and media: on the one hand, tech catastrophists believe 
that every technological advancement will put an end to human character- 
istics (“Social media destroy human relations!”); on the other hand, tech 
enthusiasts consider new technologies as superpowers helping humans 
doing something previously impossible for them. Digital humanism posi- 
tions itself in an intermediate way between these two poles: it aims to 
design a human-centered technology able to provide value to humans. In 
this, it is essentially semiotic: to provide value to humans, tech profession- 
als need to understand their (own) cultural meanings, ways of interpreta- 
tion and culture (Lagopoulos 1993: 255). 

Semiotics has provided tools to address the meaning-making and inter- 
pretations of digital communications and practices, presenting several 
examples taken from the present-day digital semiosphere, that is the appli- 
cation of the semiosphere to the dynamics of today’s globalized digital 
media systems (Hartley et al. 2021). Focusing on how people make mean- 
ings and exchange value within the digital semiosphere, semiotics has pro- 
vided innovative approaches to digital technology capable of negotiating 
increased decision-making power in the users. Semioticians have largely 
focused on digital media and communications (e.g. Cosenza 2014; Ferraro 
and Lorusso 2016). Giacomazzi (2022) proposed a methodological frame- 
work for integrating semiotics into the cultural analysis of digital media. 
He addressed the textual and experiential dimensions of digital media and 
discourses, considering the processes of digital textual production and 
translation, where users constitute themselves in relation to others. 

Recently, semiotics has expanded to include in its study the practices 
of digital culture and the digital technologies that withstand them. Some 
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research has empirically contributed to the design of digital solutions by 
proposing design methodologies taking users’ meanings and interpreta- 
tions in mind (e.g. Sanna 2020; Dall’Acqua and Bellentani 2023). San- 
tangelo and Leone (2023) edited a collection of essays proposing semiotic 
perspectives on the impact that artificial intelligence has on how human 
societies and cultures manage meaning. A line of semiotic research has also 
appeared focusing on specific digital marketing practices, such as digital 
storytelling, web design and usability, user interface and experience, online 
virality, gamification and video gaming, and branding digital communica- 
tion (Marino 2022b; Cosenza and Bianchi 2020). Digital Age in Semiot- 
ics & Communication — the journal from the Southeast European Centre 
for Semiotic Studies at the New Bulgarian University founded by Kristian 
Bankov — aims to provide a space for researching phenomena in the digital 
world. Among others, topics have included gastronomy (Mangiapane and 
Bankov 2021; Stano 2021), love and sex (Leone 2019b; Soro 2019) and 
brand communication in digital culture (Kartunova 2018). 

Much interdisciplinary research has been done on the face in the digi- 
tal era, mainly carried out by semioticians within the ERC research pro- 
ject FACETS. This research has focused on practices of face exhibition 
in social networks and in new visual technologies for face recognition, 
detection, representation and manipulation (Leone 2018c). At the same 
time, this research community has advanced the understandings of the 
semiotics of digital era looking at face-related topics such as digital dat- 
ing (Leone 2019b; Soro et al. 2021), deep fake (Leone 2022a; Santangelo 
2022; Gramigna 2022c), metaverse (Gramigna 2022b; Giuliana 2022a 
Faccia; Barbotto 2022), e-learning (Leone 2021a), digital cosmetics (Leone 
2020a), digital forensic (Leone 2020b), selfies (Santangelo 2020; Surace 
2020c; Leone 2019a ), virality (Leone 2020e; Marino 2022b), emojis and 
memes (Marino 2022a), and software for the creation of digital characters 
(Giuliana 2022b). 

Leone (Forthcoming b) addressed the impact of digital technology on the 
presence and meaning of the human face in the city. This topic relates to the 
second section of a roadmap that the author developed for the interdiscipli- 
nary study of the face and facial representations in urban contexts, which 
is divided into four sections: (i) “Faces in the city” relates to the presenta- 
tion and representation of faces in urban space; (ii)“Faces around the city” 
investigates digital facial assemblages as processed by facial technology; 
(iii) “Faces on the city” refers to old and new practices of face inscrip- 
tion on urban surfaces; and (iv) “Faces of the city” looks at the metaphor 
of cities as organisms endowed with individual faces (Leone forthcoming 
b). Between the first and the second category, Thibault and Buruk (2021) 
adopted a transurbanist framework to explore the relationship between 
faces, technology and urban spaces. The authors looked at issues of iden- 
tity and self-expression, surveillance and practices of resistance, focusing 
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on strategies of covering and modifying the face to avoid facial detection 
and recognition. 

Semiotic research has also appeared on the digital representations of the 
city in cinema and online social media networks. For example, Dimitriadis 
(2018) analyzed how the filmmaker Tasos Boulmetis combined digital ele- 
ments to reconstruct a virtual experience of his own sense and memory 
of Istanbul through computer-generated imagery (CGI): digitally recon- 
structed, the urban landscape becomes a hybrid space made of historical 
and personal memories, experiences, and emotions. Here digital technol- 
ogy is used to bring to life urban spaces from the past, as well as it impacts 
our experience of reality, our cognitive, axiological, emotional and prag- 
matic dimension. Bellentani and Arkhipova (2022) presented a biosemiotic 
approach to the study of the built environment, its representations and 
practices in social media. They identified five main topics representing the 
interplay between the natural and the built environment: the interaction of 
nature and architecture, urban parks, urban agriculture, digital environ- 
mentalism, and ecotourism. 

Thibault (2020) edited a collective book including semiotic contribu- 
tions on urban gamification. The semiotic interest on this topic has three 
reasons: firstly, representations of the city in Western culture have always 
been objects of play and illusion; secondly, there are many successful games 
which has the city as their object (Lego, Meccano, Monopoly, video games 
such as SimCity, and so on); thirdly, play and gamification are now perva- 
sive practices characterizing many different aspects of our life in the city: 
transport, shopping, sport, tourism, eating and so on (Volli 2020). 

Despite this significant and still growing research on various aspects 
of the digital culture in the city — together with the compelling semiotic 
research exploring the relationship between space, memory and identity 
(e.g. Mazzucchelli 2010; Pezzini 2009; Machin and Abousnnouga 2013; 
Bellentani and Panico 2016; Violi 2017; Panico 2019) — there is still not 
much semiotic research on digital memorials and commemorative practices. 
This chapter aims to overcome this limitation by developing a semiotic- 
oriented framework for the design of digital memorials to ensure that digi- 
tal technology remains centered on human needs and meanings. 


3. A semiotic-oriented framework for the design of digital 
memorials 


Digital memorials have existed since the advent of the Internet, which has 
become both a “vehicle of memory” to collect, preserve and display traces 
of the past and a “supplement” to traditional forms of commemorations 
(Haskins 2007: 401). However, digital memorials depend on four catego- 
ries that go beyond the mere preservation and translation of traditional 
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memorials into digital forms and that, to date, have been little explored: 
(i) the interplay between designers and users; (ii) the interplay between the 
purpose of commemoration and contents; (iii) the cultural and political 
context; (iv) the sacred and secular dimensions. This section analyzes each 
of these categories. 


3.1. The interplay between designers and users 


As their physical counterparts, digital memorials are designed by one or 
more designers: these can be institutions, individuals or groups of people 
(Moncur and Kirk 2014: 966). Inevitably, the memorial is intrinsically 
linked with the designers’ intentions and narratives that decide what 
contents to create to present the purpose of commemoration, a selection 
of what is to be remembered (Tamm 2013). However, digital memorials 
are always open to a myriad of interpretations depending on the points 
of view that the users take towards them. Despite design strategies 
being available to entice users along a specific interpretation, designers 
do not have complete control over the interpretations of digital memo- 
rials and thus users interpret them following their opinions, beliefs and 
feelings. Therefore, the interpretations of digital memorials lie at an 
intermediate point between the designers’ and the users’ interpretations 
(Eco 1990). 

Digital technologies and platforms offer and stimulate the collabora- 
tion of different users and are thus far from traditional authorial logic 
and constraints. Memorialization of deceased people through digital and 
social media platforms is designed to be interactive and to stimulate the 
active contributions of users in terms of content generation, reactions and 
commenting. Offline vernacular commemorative practices such as rituals, 
parades and performances have typically been of an ephemeral and tem- 
porary nature (Haskins 2007); with the digital, everyone can participate 
in and generate memorials that can reach a potentially limitless audience 
and that can last long. For this reason, the digital has helped to overcome 
the division between official and vernacular memorials and increase their 
participatory nature: 


The line between official and vernacular memory practices, however, 
is becoming blurry, as designers, museum professionals and art critics 
begin to ponder how “permanent” memorials might engage their popu- 
lar audiences instead of imposing on them the ossified values of political 
and cultural elites. . . . Instead of only official accounts disseminated 
by mainstream media and the government, all kinds of stories can now 
become part of an evolving patchwork of public memory. Formerly lim- 
ited in time and space, ephemeral gestures can be preserved in still and 
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moving images, ready to be viewed and replayed on demand. Previously 
banished to dark storage rooms, mementos left at memorial sites can 
be displayed for all to see. The boundaries between the official and the 
vernacular, the public and the private, the permanent and the evanescent 
will cease to matter, for all stories and images will be equally fit to rep- 
resent and comment on the past. 

(Haskins 2007: 404-405) 


A notable example of the interplay between designers and users can be 
found in the National September 11 Memorial. The memorial normally 
represents the voice of the Foundation responsible for it, but it also 
includes a website, a mobile app and an online audio repository with oral 
histories (Moncur and Kirk 2014; see also Section 4). A single organiza- 
tion designs its functions and reviews its content, but the emails, texts, 
pictures and audio are from those affected by the 9/11 attack. Digital 
memorials are thus multimedia texts constantly evolving, a product of 
the collaboration of different users, far from traditional authorial logic 
and constraints. Yet their promise of diversity and participation needs 
critiques and explanations. Firstly, “the scale of collecting increases in 
inverse proportion to our depth perception” (Gillis 1994: 15). Data are 
growing dramatically and exponentially due to the increasing number of 
data-generating devices. While digital devices record every piece of infor- 
mation that can be instantly recalled through advanced search systems, 
the individual need to remember fades away if not stimulated by partici- 
patory engagement (Haskins 2007: 407). Secondly, online platforms have 
been largely shaped by commercial and business interests, stimulating 
individualism and spectatorship rather than meaningful engagement and 
participation (Haskins 2007). Moving from technocratic and commercial 
discourses proposing to treasure personal memories, stories and sensi- 
bility of users in a social and participatory manner — against logics of 
commodification, control, standardization and commercial exploitation 
-— can help to make digital memorials more friendly to popular participa- 
tion. Thirdly, despite allowing participation of potentially limitless audi- 
ences, the digital creates new divides and inequalities around the access 
to information and resources (see Van Dijk 2006). Finally, the tendency 
towards selective nature of national politics of memory and identity 
(Tamm 2013) also manifests itself in the digital semiosphere: government 
institutions and its affiliates can decide on whose heritage will be digital- 
ized and whose not, selecting relevant audiences and granting (or less) 
access to specific online memorial contents. This topic is closely related to 
the issues of security and how digital memorials are misused and hacked 
during political conflicts and wars (Rutten et al. 2013). Cyber-attacks are 
also often used during traditional memorial wars: for example, a series 
of cyber-attacks targeted several Estonian websites following the removal 


Evanescent faces 199 


and relocation of a Soviet memorial in Tallinn, the so-called Bronze Sol- 
dier (Bellentani 2021). 

While discussing the interplay between designers and users, “digital self- 
memorialisation” (Moncur and Kirk 2014: 967) should also be considered: 
anyone can author their own memorial pre-mortem influencing the way 
they will be remembered. Specific services, such as the plugin If I Die by 
Facebook, help users to record a video or text testimony that is published 
online after the user’s death. 


3.2. The interplay between the purpose of commemoration 
and the contents 


Digital memorials are meant to commemorate and remember an individual, 
a group of people, an event or a place. Besides the actual purpose of com- 
memoration, the contents of the memorial are subjected to the designers’ 
control and to the users’ interpretations and reuses. The historical narra- 
tive which memorials represent is then a construction, inevitably selective 
and subjective. Overall, history is not a repository of past facts, events and 
identities, and it does not have an ontological status: historical narratives 
are not a passive delivery but powerful media able to establish specific dis- 
courses on the past, present and future (Ricoeur 2000). Undoubtedly, when 
speaking of institutional memorials, shaping historical narratives facilitates 
the contemporary exercise of power by promoting the dominant world 
views and determining the social dynamics of inclusion and exclusion of 
those who design them (Bellentani 2021). Selection, editing and omission 
come into play in the (re)writing of history and thus in its representation in 
digital memorials. In brief, while the purpose of commemoration is guided 
by past circumstances and the urge to commemorate, the actual contents 
of the memorials represent the designers’ perspectives. Users can always 
refashion the designers’ intention in multiple ways through their interpre- 
tations and practices. 


3.3. The cultural and political context 


Digital memorials cannot be analyzed separately from the cultural context. 
Culture shapes the designers’ and the users’ interpretations and influences 
the ways they use them. 


The ubiquity of the memorial is bound up with cultural modes of prac- 
tice, which dictate how we might orient to memorials, how we acknowl- 
edge them, sanctify them and interact with them. . . . It is also influenced 
by how individuals and groups understand the experience of bereave- 
ment, even within the same cultural group. 

(Moncur and Kirk 2014: 965) 
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In turn, digital memorials convey cultural meanings contributing to the 
shaping and reshaping of the digital semiosphere. The semiotic concept of 
culture is structured in different levels of organization. Eco (1984) divided 
culture into the global level, which included the cultural knowledge as a 
whole, and the local level, which defined the routinised ways to use that 
knowledge. He introduced the notion of encyclopedia to indicate the stock 
of shared signs that interpreters use during their interpretative processes: at 
the global level, the encyclopedia contained all the potential interpretations 
circulating in culture; at local, there was the routinised set of instructions 
to interpret specific portions of the sociocultural space, that is “encyclope- 
dic competence” (Eco 1984: 2-3). 

Torop (2002: 593) defined culture as a “mechanism of translation” char- 
acterized by the constant interaction between its global level and its local 
manifestations. The specificity of a culture originated from the friction 
between these two levels. Lotman (2005) described this process through 
the center-periphery hierarchy, one of the mechanisms for the internal 
organization of the semiosphere. Central cultures continuously attempted 
to prescribe conventional norms to the whole culture. Most members of 
culture embodied these norms and perceived them as their own reality. 
However, peripheral cultures could always arise and variously refashion 
the central norms. In doing so, peripheral cultures were vital sources for 
the definition and the development of the central culture itself. The center- 
periphery hierarchy by Lotman can be useful in explaining the interpreta- 
tive dynamics of digital monuments. Memorials “possess a powerful and 
usually self-conscious symbolic vocabulary or iconography that is under- 
stood by those who share a common culture and history” (Hershkovitz 
1993: 397). Every culture defines its own digital design models and prac- 
tices to convey this symbolic vocabulary in space. But the ways in which 
memorials are designed can elicit a range of different interpretations and 
practices from the users. Culture thus consists of different interpretative 
communities (Yanow 2000), each one having its way to frame social reality 
based on specific cultural traits, political views, socio-economic interests as 
well as contingent needs. 

Institutional digital memorials (Section 4) become part of those elements 
of culture that are defined as canonical by an institutionally sanctioned 
group of experts and professionals. They are thus important tools for the 
national politics of memory and identity; that is, the elite attempts to pro- 
mote a uniform national memory and identity through legal, institutional, 
commemorative and monumental means (Tamm 2013). As such, they are 
particularly useful in contentious political circumstances as they serve to 
“strengthen support for established regimes, instilled a sense of political 
unity and cultivated national identity” (Whelan 2002: 509). National elites 
can manipulate memory and identity for political purposes, helping to 
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promote a uniform national memory and reinforce sentiments of national 
belonging. Physical and digital memorials are tools helping elites to legiti- 
mate the primacy of their political power and to set their political agen- 
das. However, they embody the agency of generations and assume different 
functions in different time periods: memorials legitimizing elite power can 
turn into digital spaces for resistant political practice, creativity and rec- 
reation. As opposed to physical ones, digital memorials can adapt and be 
updated more easily than the physical one, and digital technology can help 
in dealing with contested physical memorials (Section 5). 


3.4. The sacred and secular dimensions 


Blue plaques in the United Kingdom are permanent markers installed in 
public places to remember the link between a location and a famous per- 
son or an event, for example, the plague “Freddy Mercury (Fred Bulsara) 
1946-1991 Singer and Songwriter lived here” at 22 Gladstone Avenue, 
London. These are secular memorials working as a “signifier of impor- 
tance” (Moncur and Kirk 2014: 971) of a place or building. Equally, the 
material possessions can become memorial tools to remember individu- 
als after their death. Personal belongings can have an emotional content 
and be important “markers of significance” (Moncur and Kirk 2014: 971). 
These can be clothes, personal object and elements closely related to the 
face, as the face is a unique and recognizable sign of identity belonging to 
the deceased (Peraica 2021): eyewear, hats, face cosmetics and so on. 

These memorial elements are important yet mundane. Other memori- 
als can have a larger spiritual dimension and can be designed to have a 
higher emotional impact on the users, as their physical counterparts such 
as tombs, crypts, cenotaphs, precinct, memorials to soldiers and battalions, 
and so on. Digital technologies and religious traditions have often been 
regarded as distinct, but today this gap is being reduced: digital platforms 
are being permeated with more and more sacred contents, and religious 
practices are increasingly becoming digital. An iconic moment demonstrat- 
ing this is in January 2019 when Pope Francis presented the app Click to 
Pray during the Sunday Angelus prayer in St. Peter’s Square. 


3.5. The framework for the design of digital memorials and 
methodological note 


In summary, digital memorials are multimedia texts brought about by a 
designer’s intention but constantly evolving due to the reworking and col- 
laboration of multiple users. Besides their purpose of commemoration, 
their contents are curated by designers which decide what memorial ele- 
ment to display and which memorial practices to stimulate. Inevitably, 
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users differently interpret and use digital memorials according to their spe- 
cific cultural traits, political views, socio-economic interests and contin- 
gent needs. Culture shapes the designers’ and the users’ interpretations and 
practices. In turn, digital memorials convey cultural meanings contributing 
to the shaping and reshaping of the digital semiosphere. They can have 
secular or sacred aspects. 

As our life becomes more and more digital, the categories presented here 
describe how digital technology intersects with a fundamental aspect of our 
life: how we memorialize. The intention was to present a semiotic-oriented 
framework including the described categories to consider when designing 
digital memorials: the interplay between designers and users, that between 
the purpose of commemoration and contents, their cultural and political, 
secular or sacred dimensions. The next sections will use this framework 
to analyze three kinds of digital memorials and commemorative practices 
centered around the face: (i) institutional digital memorials (Section 4); (ii) 
open, creative and playful solutions to reimagine contested monuments 
and memorials (Section 5); and (iii) institutional and vernacular digital- 
native commemorative practices (Section 6). 

These research objects were identified by conducting one year of digi- 
tal ethnography and studying the research literature. Digital ethnography 
(Miller 2018) on relevant websites and social media was used to gather 
empirical data on digital memorials. It also aimed to analyze the online 
behavior and digital communications of a diverse sample of online com- 
munities to practices of commemoration, remembering and mourning. 
Real-life cases were taken from the research literature on memorial tech- 
nologies (Brubaker et al. 2013; Moncur and Kirk 2014; Klastrup 2014; 
Gotved 2015). 


4. Institutional digital memorials 


Official memorials are designed by an institution to help users to “make 
sense of national tragedy” by sharing a public sense of loss (Cohen and 
Willis 2004: 591). They can commemorate and remember victims of war, 
ethnic cleansing, mass violence, terrorism, or natural disaster. While doing 
so, they inevitably represent the intentions of the designers. For this rea- 
son, as physical ones, digital memorials have both commemorative and 
political functions (Bellentani 2021). While articulating specific historical 
narratives, they convey the world views of those who design them, rep- 
resenting a whole set of meanings, identities and events while concealing 
others. Therefore, they define what and who is to be remembered of the 
past and how. 

The National September 11 Memorial is frequently cited as a prime exam- 
ple of institutional digital memorial, which aims to represent a national 
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loss through a consistent narrative (Moncur and Kirk 2014). Constantly 
updated, new functions are added from day to day by the Foundation that 
is responsible for it. A single organization thus designs its functions and 
reviews its content. The memorial includes a website, a mobile app and 
an online audio repository with oral histories of those affected by the 9/11 
attack (Moncur and Kirk 2014), yet it is the Foundation’s responsibility to 
weave these materials (emails, texts, pictures, voices and so on) into a sin- 
gle narrative. Conversely, the September 11 Digital Archive deliberatively 
represents multiple and sometimes opposite voices from the users, creating 
a more heterogeneous memorial (Moncur and Kirk 2014). From its old- 
fashioned home page, we can read that the memorial includes “more than 
150,000 digital items, a tally that includes more than 40,000 emails and 
other electronic communications, more than 40,000 first-hand stories, and 
more than 15,000 digital images” (https://911digitalarchive.org/). 

Institutional memorials and museums can integrate both physical and 
digital elements. In the 9/11 Museum in New York, there is a memorial 
corridor displaying the pictures of the faces of the victims on its walls. 
Those pictures are also included in interactive displays: by clicking on 
them, one can access the personal data and stories of the 9/11 victims. The 
victims’ face on the digital displays thus becomes an inter-face to enter 
their private life with a purpose of commemoration and mourning. Invis- 
ible to the subject, the face is normally an interface with the others, a tool 
for intersubjectivity (Leone 2021c). Mostly smiling faces are represented. 
People are mainly in elegant clothes or in their professional uniform: one 
can easily recognize the firefighters, the cops, the guardians, the cook, the 
waiters and so on. 

A similar example of this is the digital Wall of Faces honoring every per- 
son whose name is inscribed on the physical Vietnam Veterans Memorial in 
Washington, DC, through pictures of their faces. In these cases, it becomes 
clear the face is not only an interface but a text through which personality, 
identity and character are signified (Leone 2021c). In the digital Wall of 
Faces, the text is interactive and arranged by multiple authors since it allows 
family and friends to share memories, post pictures and connect with each 
other. As part of the 9/11 Memorial and Museum, Postcard is a memorial 
where faces become physical. The memorial honors Staten Island residents 
killed in the 9/11 terrorist attack and the 1993 World Trade Centre bomb- 
ing. On the memorial, victims are memorialized on individual postcards, 
remembered through their unique profile silhouette (Figure 12.1). Faces, 
cast in store and highlighted by light passing through them, here become 
symbols of distinctiveness of the remembered victims. 

In Tallinn, Estonia, there is a case of landscape memorial incorporat- 
ing digital and physical elements outside the boundaries of the museum: 
the Victims of Communism 1940-1991 memorial, unveiled in 2018 to 
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Figure 12.1 The victims’ profile silhouettes on the memorial Postcards by archi- 
tect Masayuki Sono in Staten Island, New York. Copyright: Creative 
Commons. 


commemorate Estonia’s victims of communism during Soviet rule. The 
memorial consists of three main parts, two physical and one digital: a 
memorial corridor consisting of two walls bearing plaques with the names 
and other data of over 22,000 Estonians murdered during the Soviet rule, a 
ceremonial square and the online database to search for information regard- 
ing the victims, compiled and administered by the Estonian Institute of 
Historical Memory (Bellentani 2021). The database can be accessed online 
and from a display on the site of the memorial (Figure 12.2). Research on 
the victims is continuous, and the list of names is never final: new names 
are periodically added to both the online database and the memorial walls. 
Through it, everyone can make a new entry for a name to be added to 
the wall and give feedback on corrections and improvements. Here digital 
technology allows users to update the electronic database of the memo- 
rial, thus widening public participation. The online database of victims is 
accessible to all, anytime and from anywhere: the users of the memorials 
are thus not passive spectators but active learners and co-creator of it (Bel- 
lentani 2021). 

The Digital Memorial Cemetery by designer Hadas Arnon is a peculiar 
case of physical-digital memorial. Online identities and digital data survive 
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Figure 12.2 The interactive display to access the online database of the Victims of 
Communism Memorial in Tallinn. Copyright: Photograph taken by 
the author on 26 February 2022. 


the dead, leaving family and friends with difficult decisions on the appro- 
priate ways to treat them: Arnon’s digital cemetery holds them on memory 
sticks and allows the loved ones of the deceased to access them, as in a real 
cemetery which it virtually resembles (Figure 12.3). The tombstone sticks 
are gathered into a memorial archive where people can find them through 
a computer search. 

QR codes are other tools between the digital and the physical dimen- 
sions used to memorialize the dead. Online services providing QR codes 
as digital extensions to headstones, grave markers, urns, memorial benches 
and plaques can be found. By scanning them with smartphones, family 
and friends can access a password-protected or public webpage with the 
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Figure 12.3 Hadas Arnon’s Digital Memorial Cemetery. Copyright: photography 
from www.designboom.com/project/digital-memorial-cemetery/. 


memories of the deceased for many years, shareable with anybody unable 
to visit the grave in person. Despite the Covid-19 pandemic causing a resur- 
gence of the QR code as a touchless technology to display information, its 
usage as digital memorial is still evolving (Gotved 2015). The project Scan 
Memories used to work with a similar logic: it created geographically dis- 
tributed physical memorials in meaningful locations which gave access to 
digital contents (Moncur and Kirk 2014). 

Cases of digital and physical memorials have been analyzed in the exten- 
sive literature on digital (and digitalization) of cultural heritage. In recent 
years, there have been many research projects at the intersection of comput- 
ing, digital technology and humanities aimed at digitalizing tangible and 
intangible cultural heritage. A CORDIS Results Pack specifically focused 
on the digitalization of cultural heritage collects research results from 16 
EU-funded projects designing digital technologies to protect and preserve 
Europe’s cultural heritage (e.g. ARCHES and DigiArt). Some projects are 
in between the institutional and the vernacular dimension: for example, 
during the Covid-19 pandemic, historians and digital humanists initiated 
several projects aiming to create digital memory banks by inviting people 
to upload on dedicated online platforms personal stories, pictures, videos 
connected to their life during the pandemic and its impact on everyday life 
(e.g. Zumthurm and Krebs 2022). These projects have different aims and 
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objectives, rely on various methodologies and use different technologies. 
Yet they are based on an overarching narrative seeing digitalization as a 
powerful means to ensure the preservation of Europe’s cultural heritage 
from natural and human-derived threads to ease the access and make avail- 
able cultural artifacts for digital-native future generations, enhancing par- 
ticipation and inclusivity. However, these projects have often started from 
the power of technique and digitalization, rather than fostering a more 
humanistic idea of memory by designing digital solutions that consider its 
multiple nature, the coexistence of different interpretations of the past, spa- 
tial representations and practices of commemoration. For this reason, the 
digitalization of cultural heritage is still in a transitional phase: many pro- 
jects remain rather abstract, away from the memorial needs of users and 
far from fostering their engagement and innovative use. Hence there is still 
a gap between digital and traditional memorials, a gap that is unknown to 
many other aspects of our life. 

Digital technology has also been used for a long time in monuments’ 
design, from digital master plan proposals to digital techniques underpin- 
ning their construction. Used in urban planning documents and online, 
digital representations have appeared to generate “models” of memorials 
in the future (Remm 2016: 35). Digital archaeology has also used digi- 
tal photography, 3D reconstruction, augmented and virtual reality, digital 
twins and GSI to collect archaeological data to avoid invasive or complex 
fieldworks, aiding the preservation of archaeological artifacts (e.g. Forte 
and Campana 2016). Video games have sometimes been used to help the 
restoration of damaged monuments. When Notre Dame de Paris was dev- 
astated by a fire in April 2019, it seemed reasonable that the designers 
of the video game Assassin Creed, which carefully recreated an accurate 
depiction of the cathedral, could contribute to its reconstruction. Eventu- 
ally the plan was not realized, since the digital construction of the cathedral 
was not as accurate as to follow logics related to the gameplay. 

As the “face” of monumental buildings and architectures, facades have 
been repeatedly used as canvas for digital and video mapping, light shows 
and laser game, often accompanied by digital sound. These projects can 
have site-specific connections with the building: for example, in 2022 the 
Basilica of San Petronio in Bologna, Italy, hosted a video mapping show on 
its incomplete façade showing all the plans to rebuild it, from the Middle 
Age up to today. Digital sound was added for the nights when the show 
was on. In other cases, digital and video mapping can have an artistic or 
political aim. During the 21st climate conference held in Paris in 2015, 
Naziha Mestaoui’s project Virtual Forests was projected onto the most 
famous monuments in Paris, Rome, London, Berlin and Brussels: the pro- 
ject allowed people to create through an app a digital tree that they will 
be able to see growing on monuments following their heartbeat, detected 
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through a sensor connected to the smartphone. The same technologies 
have also been used for more general purposes, such as Christmas light- 
ings. Digital projecting has also been used during protests and resistant 
practices as the projecting of the faces of Afro-American heroes on the 
pedestal of contested Confederate statues (Section 5). 

Digital technology project also showed virtual memorials and monu- 
ments that were never built or completed. An example can be found in the 
digital recreation of Democracy Uniting the World, a never-built memorial 
in San Pedro, California. Drawing on the idea of spreading democracy 
around the world developed during the 1950s, this monument was planned 
to showcase democracy in a grandiose way so as for all the people to see it. 
The monument would have been 480 feet tall, on a 46-foot-high pedestal, 
topped with a bronze globe 125 feet in diameter, with three figures repre- 
senting statesmanship, education and art. Again, a digital reconstruction 
shows the original plan of Mount Rushmore in Keystone, South Dakota, 
where figures of the presidents included their bust, and a timeline of Ameri- 
can history was inscribed next to them. In narrative semiotics, the concept 
of virtuality is central, referring to the values that underlie narratives and, 
especially, how these values establish alternative models of functioning of 
the narrated world, characters and actions. Far from being mere fictional, 
virtual narratives function as a primary modeling system of our experience 
of reality (Ferraro 2015), and so it is the digital reconstructions of never- 
built or incomplete memorials. 


5. Open, creative, playful solutions for contested monuments 
and memorials 


Once designed or published online, digital memorials become “social prop- 
erty” (Hershkovitz 1993: 397), and thus they can be used and reinterpreted 
in ways that are different from the designers’ intentions (Hay et al. 2004). 
Firstly, digital memorials embody the agency of generations and assume 
different functions as time passes. Originally designed to preserve the mem- 
ory of specific events and identities, digital memorials of a bygone era can 
remain static in their out-of-date digital forms. In other situations, digital 
memorials representing outdated cultural values or reflecting the ideology 
of a bygone regime can become contested. 

Digital technologies have also been used on both institutional and ver- 
nacular levels to provide open, creative and playful solutions for contested, 
physical monuments. These solutions are meant to rethink the contested 
meanings of monuments and to reimagine them as venues of encounter 
and active learning, rather than spaces inspiring controversy. Most of 
these practices are centered around the face: the most striking example 
of this is during the 2020 George Floyd protests when artist Dustin Klein 
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used digital projection to show the faces of Afro-American citizens killed 
at the hands of police, as well as Afro-American men and women who 
have shaped American history. Faces were projected on the pedestal of the 
Robert E. Lee Monument, the only remaining Confederate monument in 
Monument Avenue Historic District in Richmond, Virginia (Figure 12.4). 
Through this project named Reclaiming the Monuments, Klein aimed to 
change the meanings of the statue to the commander of the Confederate 
States Army, translating it into a space of Black empowerment. In the art- 
ists’ words: 


From what started off as a memorial to police violence has now become 
a theme of Black empowerment. . .. We have the attitude that the statue 
has cast dark energy towards Black people in this space since 1890. We 
started striving to find content that we felt could re-contextualize some 
of the negative energy with positive Black energy. We started projecting 
historical Black figures. . .. We also worked to include Black visual art- 
ists such Basquiat, Gordon Parks, Jacob Lawrence, and other national 
and regional artists. We plan on continuing to attend protests and work 
with the [Black Lives Matter] movement to create new appropriate con- 
tent on a daily basis. Pd like to continue the projections until the statue 
is removed. 

(Klein in Stewards 2020) 


Another digital project of historical reclamation is the Hidden Black Sto- 
ries, which allows visitors of Trafalgar Square in London to discover the 
stories of Black British who served at the Battle of Trafalgar via an immer- 
sive lens through their smartphone. The technology, realized by Snapchat 
and Black Cultural Archives, allows visitors to see the text and image car- 
ousels offering an instant Black history lesson. In all these cases, digital 
technology is used to add the missing elements of minority or forgotten 
histories to the memorial landscape which had not included them until 
now. At the institutional level, digital signage has often been referred to as 
a simple tool to recontextualise contested monuments without removing 
it from their original location, easily giving new information context to 
them. Not always a contested monument is removed, for lack of economic 
resources, religious reasons, potential ethnical threats — or simply because 
the monument looks controversial only to a small part of society. In this 
case, adding signage can help to give more context to the position of insti- 
tutions towards the identities and meanings it represents. 

Physical monuments and memorials are largely represented in social 
media communications and images. Vernacular commemorative practices 
and rituals can be reported on social media feeds and later shared and 
commented on. Moreover, it is common for tourists to take pictures of 
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Figure 12.4 The projection Reclaiming the Monuments on the Robert E. Lee Mon- 
ument statue. Copyright: photograph from Dustin Klein’s Instagram 
profile. 


memorials that are part of the city’s cultural heritage and share them online. 
The tourist practices can create controversies when they do not recognize 
or respect the publicized sentiments of mourning of memorials. The Berlin 
Holocaust Memorial, for example, has often brought together playful and 
leisure practices, as well as a general consumption of its space. Institu- 
tions and the public see this as disrespectful practices, far from the actual 
purpose of commemoration of the memorial. The writer Shahak Shapira 
challenged these practices through the artistic project Yolocaust (https:// 
yolocaust.de/), a website with photo collages combining the visitors’ selfies 
and photographs from German concentration camps. As a result, people 
end up smiling in a mass grave or jumping over piles of corpses. The project 
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has helped to reflect on commemorative culture and the gap between the 
publicized purpose of memorials and the interpretations and practices of 
users, who always resemantize the space of memorials through a myriad of 
different practices. 


6. Institutional and vernacular digital-native commemorative 
practices 


As research on the design of digital memorials abounds, there is less explo- 
ration of the modes of commemorating and mourning by way of digital 
platforms and social media, at both the institutional and vernacular levels. 
Digital-native commemorative practices originate online through digital 
communications and tools that are meant to stimulate them. The creation 
of online funeral services and rituals is a typical example of institutional 
digital-native commemorative practice. The state funeral of Elizabeth II 
on 19 September 2022 was a mass event globally broadcasted and one 
of the United Kingdom’s most-watched television broadcasts. During the 
days following the death of the Queen, the royal website and many other 
institutions set up an online book of condolences where people could post 
their grieving thoughts. Some allowed users to register for a newsletter 
summing up the collected commemorative material or to see from where 
commemorative posts come from through interactive maps. Still today on 
the website of the Church of England, it is possible to submit a message 
to an online book of condolence and access audio prayer and liturgical 
resources. All these constantly updated institutional platforms expect users 
to interact and react with their contents, as well as to generate new ones. 
Other examples include fully digital institutional memorials, for example 
the Palestinian Holocaust Memorial Museum, a virtual exhibit entirely cre- 
ated on Second Life by the US Holocaust Memorial Museum. 

Vernacular digital-native practices include multifaceted online reactions 
and responses to news of death. Social media, forums and online groups 
on WhatsApp or Facebook have increased the visibility of mourning and 
commemorative practices. Large-scale outflows of grief commonly occur 
following any notable people’s death. These are not a phenomenon typi- 
cal of the Internet era, but social media have made mourning quantifiable: 
a trend related to a celebrity is probably caused by the news of his or 
her death; that is why faking the death of a notable person (even multi- 
ple times) is a common way to get traffic to websites or social network 
accounts. Funeral services of notable people are often broadcasted online, 
as it was for Queen Elisabeth in 2022 or for the comedy performer Barry 
Chuckle in 2018 that could be followed in live reporting on the BBC news 
website. Another practice that is becoming more popular is to digitalize 
pre-digital pictures and videos, demonstrating the growing urge to create 
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individual deposits of memory to share with friends and family members 
and to transmit them to the next generations. This is made easier by the 
growing availability on the market of scanners and smartphone apps, as 
well as online services dedicated to this purpose. 

Common digital-native commemorative practices are inspired by and 
interact with physical ones, often having the face as a central element. 
As in tombstones and funeral keepsakes, the face of the dead is repre- 
sented in the digital memorial as his or her element of distinctiveness and 
recognizability. Besides this, unconventional, playful and even resistant 
practices exist. Alongside institutional display of grief, Queen Elisabeth’s 
death became the object of scorn and ridicule as ironic memes with her 
face spread all over the Internet, aiming to partly invalidate the sacred- 
ness of the moment. Civil disobedience can also grow around contested 
digital memorials that represent outdated cultural values or reflect the 
ideology of a bygone regime. Memory wars can originate online and 
are constantly fuelled during contentious times on social media, blogs 
and forums. For example, the 2007 relocation of the Bronze Soldier of 
Tallinn led to large-scale ethnic riots, one fatality, several injuries and 
thousands of arrests. During this time, the Russian Federation’s online 
media tried to justify the riots with fake news: an online blog reported 
that the memorial was cut up to allow its removal, and the picture of the 
cut memorial was shared online. Moreover, cyber-attacks were organ- 
ized targeting Estonian websites of the parliament, newspapers, banks 
and so on. 


7. Conclusions 


Digital memorials appeared in the late 1990s with the growing availabil- 
ity of the Internet and developed since then due to the increasing integra- 
tion of our lives with the digital. Commemorations and mourning have 
since diversified significantly in different digital memorials and digital- 
native practices that have contributed to democratize mourning: anyone 
connected to the Internet can share condolences and memories from any- 
where. Yet their promise of diversity and participation needs critiques and 
explanations, as explained in Section 3.1. As our “primary social image” 
(Thibault and Buruk 2021: 185), the face is a central element in digital 
memorials and commemorative practices, being it both an inter-face pro- 
viding access to the memories of the dead and a text constantly invested 
by many different interpretations and practices. The face thus becomes a 
tool for commemoration, remembrance and mourning, that is working 
“perimortem”, supporting and expanding users’ practices and experiences 
during the time surrounding death (Brubaker et al. 2013: 152). After all, 
as future studies will show, faces are central elements even in physical, 
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offline commemorative practices, as tombstones, funeral keepsakes, icons 
and public statues demonstrate. 

Unlike most other aspects of our life where digital transformation was 
more pervasive, digital memorials and commemorative practices are still 
in a transition stage between physical and digital experience. Besides the 
examples seen in Section 4 and the extensive literature on the design of 
institutional digital memorials, there are only a few cases of them that stim- 
ulate meaningful engagement and participation. Also, the incorporation of 
digital technologies in physical monuments counts only a few cases around 
the world: perhaps, the material dimension of memorials and their sacred- 
ness have so far slowed down their migration to the digital in compari- 
son to other elements of our life. One area of potential strength of digital 
technology in this context is its usage to deal with contested, physical mon- 
uments and memorials, as seen in Section 5. Section 6 showed some institu- 
tional and vernacular examples of digital-native commemorative practices. 
Among them, open, creative, playful — and even resistant — practices from 
the users always create new ways in which postmortem identities and ideas 
of the past continue to be negotiated. 

No doubt, digital memorials represent an opportunity to expand — 
socially, spatially and temporary — public commemoration and mourning. 
They allow users to gather around death, “in preparation, at the moment 
of passing, in the discovery of the death of a friend, and in the ongoing 
memorialization and grieving” (Brubaker et al. 2013: 152), to interact 
with and react to the grieving process generating new narratives for the 
dead. However, the design of actual digital memorial leans on the side of 
technology, focusing on technical functions and databases rather than pur- 
posing to treasure the personal memories, stories and sensibility of users. 
This chapter proposed a semiotic-oriented framework for the design of 
digital memorials to ensure that digital technology remains centered on 
human needs and meanings. This approach focuses on human memory, 
rather than that of databases, and puts digital technologies to the service 
of users’ memorial needs, not the other way around. As such, it promotes 
and enables a humanistic idea of memory and connection focusing on the 
four categories described in Section 3: (i) the interplay between designers 
and users, (ii) the interplay between purpose of commemoration and con- 
tents, (iii) the cultural and political context and (iv) the sacred and secular 
dimensions. This approach will create the ground for more open and pub- 
lic digital platforms against pervasive logics of commodification, control, 
standardization and commercial exploitation. 

The intention was not to be definitive in proposing this framework, but 
rather to explain the potential of the design of digital memorials and the 
breadth of digital-native commemorative practices, promoting reflections 
among both humanities and HCI research communities using real-life 
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examples. Abundant opportunities for future work exist in this topic. The 
framework we presented here is inspired by examples from the Western 
perspective. More work is needed to explore other cultural perspectives 
of digital memorials and commemorative practices. Further studies, which 
take into account the framework here presented, will need to be under- 
taken on the potential of digital technology in the field of cultural heritage 
and on heritage industries and policies. 


Note 


1 This chapter results from a project that has received funding from the European 
Research Council (ERC) under the European Union’s Horizon 2020 research 
and innovation program (Grant Agreement No 819649-FACETS; PI: Massimo 
LEONE). 
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